You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@sqoop.apache.org by ja...@apache.org on 2015/12/15 14:33:11 UTC
sqoop git commit: SQOOP-2736: Sqoop2: Document change command line
shell interface
Repository: sqoop
Updated Branches:
refs/heads/sqoop2 d35efd3c3 -> d89f64adc
SQOOP-2736: Sqoop2: Document change command line shell interface
(Colin Ma via Jarek Jarcec Cecho)
Project: http://git-wip-us.apache.org/repos/asf/sqoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/sqoop/commit/d89f64ad
Tree: http://git-wip-us.apache.org/repos/asf/sqoop/tree/d89f64ad
Diff: http://git-wip-us.apache.org/repos/asf/sqoop/diff/d89f64ad
Branch: refs/heads/sqoop2
Commit: d89f64adc80c3f0a8dd5acd538d663c74a7d4a1a
Parents: d35efd3
Author: Jarek Jarcec Cecho <ja...@apache.org>
Authored: Tue Dec 15 14:32:42 2015 +0100
Committer: Jarek Jarcec Cecho <ja...@apache.org>
Committed: Tue Dec 15 14:32:42 2015 +0100
----------------------------------------------------------------------
docs/src/site/sphinx/dev/ClientAPI.rst | 50 +++----
docs/src/site/sphinx/dev/RESTAPI.rst | 128 +++++++++--------
docs/src/site/sphinx/user/CommandLineClient.rst | 140 +++++++++----------
docs/src/site/sphinx/user/Sqoop5MinutesDemo.rst | 64 ++++-----
4 files changed, 194 insertions(+), 188 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/sqoop/blob/d89f64ad/docs/src/site/sphinx/dev/ClientAPI.rst
----------------------------------------------------------------------
diff --git a/docs/src/site/sphinx/dev/ClientAPI.rst b/docs/src/site/sphinx/dev/ClientAPI.rst
index d69fabd..046ec5b 100644
--- a/docs/src/site/sphinx/dev/ClientAPI.rst
+++ b/docs/src/site/sphinx/dev/ClientAPI.rst
@@ -34,9 +34,9 @@ Workflow
Given workflow has to be followed for executing a sqoop job in Sqoop server.
- 1. Create LINK object for a given connectorId - Creates Link object and returns linkId (lid)
- 2. Create a JOB for a given "from" and "to" linkId - Create Job object and returns jobId (jid)
- 3. Start the JOB for a given jobId - Start Job on the server and creates a submission record
+ 1. Create LINK object for a given connector name - Creates Link object and returns it
+ 2. Create a JOB for a given "from" and "to" link name - Create Job object and returns it
+ 3. Start the JOB for a given job name - Start Job on the server and creates a submission record
Project Dependencies
====================
@@ -76,13 +76,12 @@ you can get the list of all the config/inputs using `Display Config and Input Na
Save Link
---------
-First create a new link by invoking ``createLink(cid)`` method with connector Id and it returns a MLink object with dummy id and the unfilled link config inputs for that connector. Then fill the config inputs with relevant values. Invoke ``saveLink`` passing it the filled MLink object.
+First create a new link by invoking ``createLink(connectorName)`` method with connector name and it returns a MLink object with dummy id and the unfilled link config inputs for that connector. Then fill the config inputs with relevant values. Invoke ``saveLink`` passing it the filled MLink object.
::
// create a placeholder for link
- long connectorId = 1;
- MLink link = client.createLink(connectorId);
+ MLink link = client.createLink("connectorName");
link.setName("Vampire");
link.setCreationUser("Buffy");
MLinkConfig linkConfig = link.getConnectorLinkConfig();
@@ -94,21 +93,21 @@ First create a new link by invoking ``createLink(cid)`` method with connector Id
// save the link object that was filled
Status status = client.saveLink(link);
if(status.canProceed()) {
- System.out.println("Created Link with Link Id : " + link.getPersistenceId());
+ System.out.println("Created Link with Link Name : " + link.getName());
} else {
System.out.println("Something went wrong creating the link");
}
``status.canProceed()`` returns true if status is OK or a WARNING. Before sending the status, the link config values are validated using the corresponding validator associated with th link config inputs.
-On successful execution of the saveLink method, new link Id is assigned to the link object else an exception is thrown. ``link.getPersistenceId()`` method returns the unique Id for this object persisted in the sqoop repository.
+On successful execution of the saveLink method, new link name is assigned to the link object else an exception is thrown. ``link.getName()`` method returns the unique name for this object persisted in the sqoop repository.
User can retrieve a link using the following methods
+----------------------------+--------------------------------------+
| Method | Description |
+============================+======================================+
-| ``getLink(lid)`` | Returns a link by id |
+| ``getLink(linkName)`` | Returns a link by name |
+----------------------------+--------------------------------------+
| ``getLinks()`` | Returns list of links in the sqoop |
+----------------------------+--------------------------------------+
@@ -118,7 +117,7 @@ Job
A sqoop job holds the ``From`` and ``To`` parts for transferring data from the ``From`` data source to the ``To`` data source. Both the ``From`` and the ``To`` are uniquely identified by their corresponding connector Link Ids. i.e when creating a job we have to specifiy the ``FromLinkId`` and the ``ToLinkId``. Thus the pre-requisite for creating a job is to first create the links as described above.
-Once the linkIds for the ``From`` and ``To`` are given, then the job configs for the associated connector for the link object have to be filled. You can get the list of all the from and to job config/inputs using `Display Config and Input Names For Connector`_ for that connector. A connector can have one or more links. We then use the links in the ``From`` and ``To`` direction to populate the corresponding ``MFromConfig`` and ``MToConfig`` respectively.
+Once the link names for the ``From`` and ``To`` are given, then the job configs for the associated connector for the link object have to be filled. You can get the list of all the from and to job config/inputs using `Display Config and Input Names For Connector`_ for that connector. A connector can have one or more links. We then use the links in the ``From`` and ``To`` direction to populate the corresponding ``MFromConfig`` and ``MToConfig`` respectively.
In addition to filling the job configs for the ``From`` and the ``To`` representing the link, we also need to fill the driver configs that control the job execution engine environment. For example, if the job execution engine happens to be the MapReduce we will specifiy the number of mappers to be used in reading data from the ``From`` data source.
@@ -130,9 +129,7 @@ Here is the code to create and then save a job
String url = "http://localhost:12000/sqoop/";
SqoopClient client = new SqoopClient(url);
//Creating dummy job object
- long fromLinkId = 1;// for jdbc connector
- long toLinkId = 2; // for HDFS connector
- MJob job = client.createJob(fromLinkId, toLinkId);
+ MJob job = client.createJob("fromLinkName", "toLinkName");
job.setName("Vampire");
job.setCreationUser("Buffy");
// set the "FROM" link job config values
@@ -149,7 +146,7 @@ Here is the code to create and then save a job
Status status = client.saveJob(job);
if(status.canProceed()) {
- System.out.println("Created Job with Job Id: "+ job.getPersistenceId());
+ System.out.println("Created Job with Job Name: "+ job.getName());
} else {
System.out.println("Something went wrong creating the job");
}
@@ -159,7 +156,7 @@ User can retrieve a job using the following methods
+----------------------------+--------------------------------------+
| Method | Description |
+============================+======================================+
-| ``getJob(jid)`` | Returns a job by id |
+| ``getJob(jobName)`` | Returns a job by name |
+----------------------------+--------------------------------------+
| ``getJobs()`` | Returns list of jobs in the sqoop |
+----------------------------+--------------------------------------+
@@ -219,23 +216,22 @@ After creating link or job in the repository, you can update or delete a link or
+==================================+====================================================================================+
| ``updateLink(link)`` | Invoke update with link and check status for any errors or warnings |
+----------------------------------+------------------------------------------------------------------------------------+
-| ``deleteLink(lid)`` | Delete link. Deletes only if specified link is not used by any job |
+| ``deleteLink(linkName)`` | Delete link. Deletes only if specified link is not used by any job |
+----------------------------------+------------------------------------------------------------------------------------+
| ``updateJob(job)`` | Invoke update with job and check status for any errors or warnings |
+----------------------------------+------------------------------------------------------------------------------------+
-| ``deleteJob(jid)`` | Delete job |
+| ``deleteJob(jobName)`` | Delete job |
+----------------------------------+------------------------------------------------------------------------------------+
Job Start
==============
-Starting a job requires a job id. On successful start, getStatus() method returns "BOOTING" or "RUNNING".
+Starting a job requires a job name. On successful start, getStatus() method returns "BOOTING" or "RUNNING".
::
//Job start
- long jobId = 1;
- MSubmission submission = client.startJob(jobId);
+ MSubmission submission = client.startJob("jobName");
System.out.println("Job Submission Status : " + submission.getStatus());
if(submission.getStatus().isRunning() && submission.getProgress() != -1) {
System.out.println("Progress : " + String.format("%.2f %%", submission.getProgress() * 100));
@@ -262,15 +258,15 @@ Starting a job requires a job id. On successful start, getStatus() method return
//Check job status for a running job
- MSubmission submission = client.getJobStatus(jobId);
+ MSubmission submission = client.getJobStatus("jobName");
if(submission.getStatus().isRunning() && submission.getProgress() != -1) {
System.out.println("Progress : " + String.format("%.2f %%", submission.getProgress() * 100));
}
//Stop a running job
- submission.stopJob(jobId);
+ submission.stopJob("jobName");
-Above code block, job start is asynchronous. For synchronous job start, use ``startJob(jid, callback, pollTime)`` method. If you are not interested in getting the job status, then invoke the same method with "null" as the value for the callback parameter and this returns the final job status. ``pollTime`` is the request interval for getting the job status from sqoop server and the value should be greater than zero. We will frequently hit the sqoop server if a low value is given for the ``pollTime``. When a synchronous job is started with a non null callback, it first invokes the callback's ``submitted(MSubmission)`` method on successful start, after every poll time interval, it then invokes the ``updated(MSubmission)`` method on the callback API and finally on finishing the job executuon it invokes the ``finished(MSubmission)`` method on the callback API.
+Above code block, job start is asynchronous. For synchronous job start, use ``startJob(jobName, callback, pollTime)`` method. If you are not interested in getting the job status, then invoke the same method with "null" as the value for the callback parameter and this returns the final job status. ``pollTime`` is the request interval for getting the job status from sqoop server and the value should be greater than zero. We will frequently hit the sqoop server if a low value is given for the ``pollTime``. When a synchronous job is started with a non null callback, it first invokes the callback's ``submitted(MSubmission)`` method on successful start, after every poll time interval, it then invokes the ``updated(MSubmission)`` method on the callback API and finally on finishing the job executuon it invokes the ``finished(MSubmission)`` method on the callback API.
Display Config and Input Names For Connector
============================================
@@ -281,13 +277,13 @@ You can view the config/input names for the link and job config types per connec
String url = "http://localhost:12000/sqoop/";
SqoopClient client = new SqoopClient(url);
- long connectorId = 1;
+ String connectorName = "connectorName";
// link config for connector
- describe(client.getConnector(connectorId).getLinkConfig().getConfigs(), client.getConnectorConfigBundle(connectorId));
+ describe(client.getConnector(connectorName).getLinkConfig().getConfigs(), client.getConnectorConfigBundle(connectorName));
// from job config for connector
- describe(client.getConnector(connectorId).getFromConfig().getConfigs(), client.getConnectorConfigBundle(connectorId));
+ describe(client.getConnector(connectorName).getFromConfig().getConfigs(), client.getConnectorConfigBundle(connectorName));
// to job config for the connector
- describe(client.getConnector(connectorId).getToConfig().getConfigs(), client.getConnectorConfigBundle(connectorId));
+ describe(client.getConnector(connectorName).getToConfig().getConfigs(), client.getConnectorConfigBundle(connectorName));
void describe(List<MConfig> configs, ResourceBundle resource) {
for (MConfig config : configs) {
http://git-wip-us.apache.org/repos/asf/sqoop/blob/d89f64ad/docs/src/site/sphinx/dev/RESTAPI.rst
----------------------------------------------------------------------
diff --git a/docs/src/site/sphinx/dev/RESTAPI.rst b/docs/src/site/sphinx/dev/RESTAPI.rst
index 39aabc0..7929b28 100644
--- a/docs/src/site/sphinx/dev/RESTAPI.rst
+++ b/docs/src/site/sphinx/dev/RESTAPI.rst
@@ -110,12 +110,13 @@ Each input object in a config is structured below:
To send a filled config in the request, you should always use config id and input id to map the values to their correspondig names.
-For example, the following request contains an input value ``com.mysql.jdbc.Driver`` with input id ``7`` inside a config with id ``4`` that belongs to a link with id ``3``
+For example, the following request contains an input value ``com.mysql.jdbc.Driver`` with input id ``7`` inside a config with id ``4`` that belongs to a link with name ``linkName``
::
link: {
- id: 3,
+ id : 3,
+ name: "linkName",
enabled: true,
link-config-values: [{
id: 4,
@@ -353,10 +354,10 @@ Get all the connectors registered in Sqoop
}]
}
-/v1/connector/[cname] or /v1/connector/[cid] - [GET] - Get Connector
+/v1/connector/[cname] - [GET] - Get Connector
---------------------------------------------------------------------
-Provide the id or unique name of the connector in the url ``[cid]`` or ``[cname]`` part.
+Provide the unique name of the connector in the url ``[cname]`` part.
* Method: ``GET``
* Format: ``JSON``
@@ -367,7 +368,7 @@ Provide the id or unique name of the connector in the url ``[cid]`` or ``[cname]
+--------------------------+----------------------------------------------------------------------------------------+
| Field | Description |
+==========================+========================================================================================+
-| ``id`` | The id for the connector ( registered as a configurable ) |
+| ``name`` | The name for the connector ( registered as a configurable ) |
+--------------------------+----------------------------------------------------------------------------------------+
| ``job-config`` | Connector job config and inputs for both FROM and TO |
+--------------------------+----------------------------------------------------------------------------------------+
@@ -385,6 +386,7 @@ Provide the id or unique name of the connector in the url ``[cid]`` or ``[cname]
{
connector: {
id: 1,
+ name: "connectorName",
job-config: {
TO: [{
id: 3,
@@ -550,7 +552,7 @@ Get all the links created in Sqoop
link-config-values: [],
name: "First Link",
creation-date: 1415309361756,
- connector-id: 1,
+ connector-name: "connectorName1",
update-date: 1415309361756,
creation-user: "root"
},
@@ -561,7 +563,7 @@ Get all the links created in Sqoop
link-config-values: [],
name: "Second Link",
creation-date: 1415309390807,
- connector-id: 2,
+ connector-name: "connectorName2",
update-date: 1415309390807,
creation-user: "root"
}
@@ -574,12 +576,12 @@ Get all the links created in Sqoop
Get all the links for a given connector identified by ``[cname]`` part.
-/v1/link/[lname] or /v1/link/[lid] - [GET] - Get Link
+/v1/link/[lname] - [GET] - Get Link
-------------------------------------------------------------------------------
-Provide the id or unique name of the link in the url ``[lid]`` or ``[lname]`` part.
+Provide the unique name of the link in the url ``[lname]`` part.
-Get all the details of the link including the id, name, type and the corresponding config input values for the link
+Get all the details of the link including the name, type and the corresponding config input values for the link
* Method: ``GET``
@@ -610,7 +612,7 @@ Get all the details of the link including the id, name, type and the correspondi
update-user: "root",
name: "First Link",
creation-date: 1415287846371,
- connector-id: 1,
+ connector-name: "connectorName",
update-date: 1415287846371,
creation-user: "root"
}
@@ -673,7 +675,7 @@ Create a new link object. Provide values to the link config inputs for the ones
update-user: "root",
name: "testLink",
creation-date: 1415202223048,
- connector-id: 1,
+ connector-name: "connectorName",
update-date: 1415202223048,
creation-user: "root"
}
@@ -684,7 +686,7 @@ Create a new link object. Provide values to the link config inputs for the ones
+---------------------------+--------------------------------------------------------------------------------------+
| Field | Description |
+===========================+======================================================================================+
-| ``id`` | The id assigned for this new created link |
+| ``name`` | The name assigned for this new created link |
+---------------------------+--------------------------------------------------------------------------------------+
| ``validation-result`` | The validation status for the link config inputs given in the post data |
+---------------------------+--------------------------------------------------------------------------------------+
@@ -707,10 +709,10 @@ Create a new link object. Provide values to the link config inputs for the ones
}
-/v1/link/[lname] or /v1/link/[lid] - [PUT] - Update Link
+/v1/link/[lname] - [PUT] - Update Link
---------------------------------------------------------
-Update an existing link object with name [lname] or id [lid]. To make the procedure of filling inputs easier, the general practice
+Update an existing link object with name [lname]. To make the procedure of filling inputs easier, the general practice
is get the link first and then change some of the values for the inputs.
* Method: ``PUT``
@@ -726,30 +728,30 @@ is get the link first and then change some of the values for the inputs.
]
}
-/v1/link/[lname] or /v1/link/[lid] - [DELETE] - Delete Link
+/v1/link/[lname] - [DELETE] - Delete Link
-----------------------------------------------------------------
-Delete a link with name [lname] or id [lid]
+Delete a link with name [lname]
* Method: ``DELETE``
* Format: ``JSON``
* Request Content: ``None``
* Response Content: ``None``
-/v1/link/[lid]/enable or /v1/link/[lname]/enable - [PUT] - Enable Link
+/v1/link/[lname]/enable - [PUT] - Enable Link
--------------------------------------------------------------------------------
-Enable a link with id ``lid`` or name ``lname``
+Enable a link with name ``lname``
* Method: ``PUT``
* Format: ``JSON``
* Request Content: ``None``
* Response Content: ``None``
-/v1/link/[lid]/disable - [PUT] - Disable Link
+/v1/link/[lname]/disable - [PUT] - Disable Link
---------------------------------------------------------
-Disable a link with id ``lid`` or name ``lname``
+Disable a link with name ``lname``
* Method: ``PUT``
* Format: ``JSON``
@@ -773,33 +775,33 @@ Get all the jobs created in Sqoop
jobs: [{
driver-config-values: [],
enabled: true,
- from-connector-id: 1,
+ from-connector-name: "fromConnectorName",
update-user: "root",
to-config-values: [],
- to-connector-id: 2,
+ to-connector-name: "toConnectorName",
creation-date: 1415310157618,
update-date: 1415310157618,
creation-user: "root",
id: 1,
- to-link-id: 2,
+ to-link-name: "toLinkName",
from-config-values: [],
name: "First Job",
- from-link-id: 1
+ from-link-name: "fromLinkName"
},{
driver-config-values: [],
enabled: true,
- from-connector-id: 2,
+ from-connector-name: "fromConnectorName",
update-user: "root",
to-config-values: [],
- to-connector-id: 1,
+ to-connector-name: "toConnectorName",
creation-date: 1415310650600,
update-date: 1415310650600,
creation-user: "root",
id: 2,
- to-link-id: 1,
+ to-link-name: "toLinkName",
from-config-values: [],
name: "Second Job",
- from-link-id: 2
+ from-link-name: "fromLinkName"
}]
}
@@ -808,11 +810,10 @@ Get all the jobs created in Sqoop
Get all the jobs for a given connector identified by ``[cname]`` part.
-/v1/job/[jname] or /v1/job/[jid] - [GET] - Get Job
+/v1/job/[jname] - [GET] - Get Job
-----------------------------------------------------
-Provide the name or the id of the job in the url [jname]
-part or [jid] part.
+Provide the name of the job in the url [jname] part.
* Method: ``GET``
* Format: ``JSON``
@@ -843,7 +844,7 @@ part or [jid] part.
type: "JOB"
}],
enabled: true,
- from-connector-id: 1,
+ from-connector-name: "fromConnectorName",
update-user: "root",
to-config-values: [{
id: 6,
@@ -887,12 +888,12 @@ part or [jid] part.
name: "toJobConfig",
type: "JOB"
}],
- to-connector-id: 2,
+ to-connector-name: "toConnectorName",
creation-date: 1415310157618,
update-date: 1415310157618,
creation-user: "root",
id: 1,
- to-link-id: 2,
+ to-link-name: "toLinkName",
from-config-values: [{
id: 2,
inputs: [{
@@ -907,7 +908,7 @@ part or [jid] part.
type: "JOB"
}],
name: "First Job",
- from-link- id: 1
+ from-link-name: "fromLinkName"
}
}
@@ -928,9 +929,9 @@ Create a new job object with the corresponding config values.
+==========================+======================================================================================+
| ``job`` | The root of the post data in JSON |
+--------------------------+--------------------------------------------------------------------------------------+
-| ``from-link-id`` | The id of the from link for the job |
+| ``from-link-name`` | The name of the from link for the job |
+--------------------------+--------------------------------------------------------------------------------------+
-| ``to-link-id`` | The id of the to link for the job |
+| ``to-link-name`` | The name of the to link for the job |
+--------------------------+--------------------------------------------------------------------------------------+
| ``id`` | The id of the link can be left blank in the post data |
+--------------------------+--------------------------------------------------------------------------------------+
@@ -952,9 +953,10 @@ Create a new job object with the corresponding config values.
+--------------------------+--------------------------------------------------------------------------------------+
| ``driver-config-values`` | Config input values for driver |
+--------------------------+--------------------------------------------------------------------------------------+
-| ``connector-id`` | The id of the connector used for this link |
+| ``from-connector-name`` | The name of the from connector for the job |
++--------------------------+--------------------------------------------------------------------------------------+
+| ``to-connector-name`` | The name of the to connector for the job |
+--------------------------+--------------------------------------------------------------------------------------+
-
* Request Example:
@@ -986,7 +988,7 @@ Create a new job object with the corresponding config values.
}
],
enabled: true,
- from-connector-id: 1,
+ from-connector-name: "fromConnectorName",
update-user: "root",
to-config-values: [
{
@@ -1039,12 +1041,12 @@ Create a new job object with the corresponding config values.
type: "JOB"
}
],
- to-connector-id: 2,
+ to-connector-name: "toConnectorName",
creation-date: 1415310157618,
update-date: 1415310157618,
creation-user: "root",
id: -1,
- to-link-id: 2,
+ to-link-name: "toLinkName",
from-config-values: [
{
id: 2,
@@ -1063,7 +1065,7 @@ Create a new job object with the corresponding config values.
}
],
name: "Test Job",
- from-link-id: 1
+ from-link-name: "fromLinkName"
}
}
@@ -1072,7 +1074,7 @@ Create a new job object with the corresponding config values.
+---------------------------+--------------------------------------------------------------------------------------+
| Field | Description |
+===========================+======================================================================================+
-| ``id`` | The id assigned for this new created job |
+| ``name`` | The name assigned for this new created job |
+--------------------------+---------------------------------------------------------------------------------------+
| ``validation-result`` | The validation status for the job config and driver config inputs in the post data |
+---------------------------+--------------------------------------------------------------------------------------+
@@ -1096,10 +1098,10 @@ Create a new job object with the corresponding config values.
}
-/v1/job/[jid] - [PUT] - Update Job
+/v1/job/[jname] - [PUT] - Update Job
---------------------------------------------------------
-Update an existing job object with id [jid]. To make the procedure of filling inputs easier, the general practice
+Update an existing job object with name [jname]. To make the procedure of filling inputs easier, the general practice
is get the existing job object first and then change some of the inputs.
* Method: ``PUT``
@@ -1118,30 +1120,30 @@ The same as Create Job.
}
-/v1/job/[jid] - [DELETE] - Delete Job
+/v1/job/[jname] - [DELETE] - Delete Job
---------------------------------------------------------
-Delete a job with id ``jid``.
+Delete a job with name ``jname``.
* Method: ``DELETE``
* Format: ``JSON``
* Request Content: ``None``
* Response Content: ``None``
-/v1/job/[jid]/enable - [PUT] - Enable Job
+/v1/job/[jname]/enable - [PUT] - Enable Job
---------------------------------------------------------
-Enable a job with id ``jid``.
+Enable a job with name ``jname``.
* Method: ``PUT``
* Format: ``JSON``
* Request Content: ``None``
* Response Content: ``None``
-/v1/job/[jid]/disable - [PUT] - Disable Job
+/v1/job/[jname]/disable - [PUT] - Disable Job
---------------------------------------------------------
-Disable a job with id ``jid``.
+Disable a job with name ``jname``.
* Method: ``PUT``
* Format: ``JSON``
@@ -1149,10 +1151,10 @@ Disable a job with id ``jid``.
* Response Content: ``None``
-/v1/job/[jid]/start or /v1/job/[jname]/start - [PUT]- Start Job
+/v1/job/[jname]/start - [PUT]- Start Job
---------------------------------------------------------------------------------
-Start a job with name ``[jname]`` or with id ``[jid]`` to trigger the job execution
+Start a job with name ``[jname]`` to trigger the job execution
* Method: ``POST``
* Format: ``JSON``
@@ -1170,6 +1172,7 @@ Start a job with name ``[jname]`` or with id ``[jid]`` to trigger the job execut
"external-id": "job_1412137947693_0004",
"status": "BOOTING",
"job": 2,
+ "job-name": "jobName",
"creation-date": 1415312531188,
"to-schema": {
"created": 1415312531426,
@@ -1210,6 +1213,7 @@ Start a job with name ``[jname]`` or with id ``[jid]`` to trigger the job execut
external-id: "job_1412137947693_0004",
status: "SUCCEEDED",
job: 2,
+ job-name: "jobName",
creation-date: 1415312531188,
external-link: "http://vbsqoop-1.ent.cloudera.com:8088/proxy/application_1412137947693_0004/",
counters: {
@@ -1271,6 +1275,7 @@ Start a job with name ``[jname]`` or with id ``[jid]`` to trigger the job execut
"status": "FAILURE_ON_SUBMIT",
"error-summary": "org.apache.sqoop.common.SqoopException: GENERIC_HDFS_CONNECTOR_0000:Error occurs during partitioner run",
"job": 1,
+ "job-name": "jobName",
"creation-date": 1415312390570,
"to-schema": {
"created": 1415312390797,
@@ -1301,20 +1306,20 @@ Start a job with name ``[jname]`` or with id ``[jid]`` to trigger the job execut
}
}
-/v1/job/[jid]/stop or /v1/job/[jname]/stop - [PUT]- Stop Job
+/v1/job/[jname]/stop - [PUT]- Stop Job
---------------------------------------------------------------------------------
-Stop a job with name ``[janme]`` or with id ``[jid]`` to abort the running job.
+Stop a job with name ``[jname]`` to abort the running job.
* Method: ``PUT``
* Format: ``JSON``
* Request Content: ``None``
* Response Content: ``Submission Record``
-/v1/job/[jid]/status or /v1/job/[jname]/status - [GET]- Get Job Status
+/v1/job/[jname]/status - [GET]- Get Job Status
---------------------------------------------------------------------------------
-Get status of the running job with name ``[janme]`` or with id ``[jid]``
+Get status of the running job with name ``[jname]``
* Method: ``GET``
* Format: ``JSON``
@@ -1330,6 +1335,7 @@ Get status of the running job with name ``[janme]`` or with id ``[jid]``
"external-id": "job_1412137947693_0004",
"status": "RUNNING",
"job": 2,
+ "job-name": "jobName",
"creation-date": 1415312531188,
"external-link": "http://vbsqoop-1.ent.cloudera.com:8088/proxy/application_1412137947693_0004/"
}
@@ -1359,6 +1365,8 @@ Provide the name of the job in the url [jname] part.
+--------------------------+--------------------------------------------------------------------------------------+
| ``job`` | The id of the Sqoop job |
+--------------------------+--------------------------------------------------------------------------------------+
+| ``job-name`` | The name of the Sqoop job |
++--------------------------+--------------------------------------------------------------------------------------+
| ``creation-date`` | The submission timestamp |
+--------------------------+--------------------------------------------------------------------------------------+
| ``last-update-date`` | The timestamp of the last status update |
@@ -1382,6 +1390,7 @@ Provide the name of the job in the url [jname] part.
external-id: "job_1412137947693_0004",
status: "SUCCEEDED",
job: 2,
+ job-name: "jobName",
creation-date: 1415312531188,
external-link: "http://vbsqoop-1.ent.cloudera.com:8088/proxy/application_1412137947693_0004/",
counters: {
@@ -1435,6 +1444,7 @@ Provide the name of the job in the url [jname] part.
status: "FAILURE_ON_SUBMIT",
error-summary: "org.apache.sqoop.common.SqoopException: GENERIC_HDFS_CONNECTOR_0000:Error occurs during partitioner run",
job: 1,
+ job-name: "jobName",
creation-date: 1415312390570,
error-details: "org.apache.sqoop.common.SqoopException: GENERIC_HDFS_CONNECTOR_0000:Error occurs during partitioner...."
}
http://git-wip-us.apache.org/repos/asf/sqoop/blob/d89f64ad/docs/src/site/sphinx/user/CommandLineClient.rst
----------------------------------------------------------------------
diff --git a/docs/src/site/sphinx/user/CommandLineClient.rst b/docs/src/site/sphinx/user/CommandLineClient.rst
index ee08de6..acaca16 100644
--- a/docs/src/site/sphinx/user/CommandLineClient.rst
+++ b/docs/src/site/sphinx/user/CommandLineClient.rst
@@ -34,7 +34,7 @@ Sqoop client script is expected to contain valid Sqoop client commands, empty li
set server --host sqoop2.company.net
# Executing given job
- start job --jid 1
+ start job --name 1
.. contents:: Table of Contents
@@ -269,12 +269,12 @@ Show persisted link objects.
+=======================+======================================================+
| ``-a``, ``--all`` | Show all available links |
+-----------------------+------------------------------------------------------+
-| ``-x``, ``--lid <x>`` | Show link with id ``<x>`` |
+| ``-n``, ``--name <x>``| Show link with name ``<x>`` |
+-----------------------+------------------------------------------------------+
Example: ::
- show link --all or show link
+ show link --all or show link --name linkName
Show Job Function
~~~~~~~~~~~~~~~~~
@@ -286,31 +286,31 @@ Show persisted job objects.
+=======================+==============================================+
| ``-a``, ``--all`` | Show all available jobs |
+-----------------------+----------------------------------------------+
-| ``-j``, ``--jid <x>`` | Show job with id ``<x>`` |
+| ``-n``, ``--name <x>``| Show job with name ``<x>`` |
+-----------------------+----------------------------------------------+
Example: ::
- show job --all or show job
+ show job --all or show job --name jobName
Show Submission Function
~~~~~~~~~~~~~~~~~~~~~~~~
Show persisted job submission objects.
-+-----------------------+---------------------------------------------+
-| Argument | Description |
-+=======================+=============================================+
-| ``-j``, ``--jid <x>`` | Show available submissions for given job |
-+-----------------------+---------------------------------------------+
-| ``-d``, ``--detail`` | Show job submissions in full details |
-+-----------------------+---------------------------------------------+
++-----------------------+-----------------------------------------------+
+| Argument | Description |
++=======================+===============================================+
+| ``-j``, ``--job <x>`` | Show available submissions for given job name |
++-----------------------+-----------------------------------------------+
+| ``-d``, ``--detail`` | Show job submissions in full details |
++-----------------------+-----------------------------------------------+
Example: ::
show submission
- show submission --jid 1
- show submission --jid 1 --detail
+ show submission --j jobName
+ show submission --job jobName --detail
Create Command
--------------
@@ -332,16 +332,16 @@ Create Link Function
Create new link object.
-+------------------------+-------------------------------------------------------------+
-| Argument | Description |
-+========================+=============================================================+
-| ``-c``, ``--cid <x>`` | Create new link object for connector with id ``<x>`` |
-+------------------------+-------------------------------------------------------------+
++------------------------------+-------------------------------------------------------------+
+| Argument | Description |
++==============================+=============================================================+
+| ``-c``, ``--connector <x>`` | Create new link object for connector with name ``<x>`` |
++------------------------------+-------------------------------------------------------------+
Example: ::
- create link --cid 1 or create link -c 1
+ create link --connector connectorName or create link -c connectorName
Create Job Function
~~~~~~~~~~~~~~~~~~~
@@ -351,14 +351,14 @@ Create new job object.
+------------------------+------------------------------------------------------------------+
| Argument | Description |
+========================+==================================================================+
-| ``-f``, ``--from <x>`` | Create new job object with a FROM link with id ``<x>`` |
+| ``-f``, ``--from <x>`` | Create new job object with a FROM link with name ``<x>`` |
+------------------------+------------------------------------------------------------------+
-| ``-t``, ``--to <t>`` | Create new job object with a TO link with id ``<x>`` |
+| ``-t``, ``--to <t>`` | Create new job object with a TO link with name ``<x>`` |
+------------------------+------------------------------------------------------------------+
Example: ::
- create job --from 1 --to 2 or create job --f 1 --t 2
+ create job --from fromLinkName --to toLinkName or create job --f fromLinkName --t toLinkName
Update Command
--------------
@@ -373,27 +373,27 @@ Update existing link object.
+-----------------------+---------------------------------------------+
| Argument | Description |
+=======================+=============================================+
-| ``-x``, ``--lid <x>`` | Update existing link with id ``<x>`` |
+| ``-n``, ``--name <x>``| Update existing link with name ``<x>`` |
+-----------------------+---------------------------------------------+
Example: ::
- update link --lid 1
+ update link --name linkName
Update Job Function
~~~~~~~~~~~~~~~~~~~
Update existing job object.
-+-----------------------+--------------------------------------------+
-| Argument | Description |
-+=======================+============================================+
-| ``-j``, ``--jid <x>`` | Update existing job object with id ``<x>`` |
-+-----------------------+--------------------------------------------+
++-------------------------+----------------------------------------------+
+| Argument | Description |
++=========================+==============================================+
+| ``-n``, ``--name <x>`` | Update existing job object with name ``<x>`` |
++-------------------------+----------------------------------------------+
Example: ::
- update job --jid 1
+ update job --name jobName
Delete Command
@@ -406,15 +406,15 @@ Delete Link Function
Delete existing link object.
-+-----------------------+-------------------------------------------+
-| Argument | Description |
-+=======================+===========================================+
-| ``-x``, ``--lid <x>`` | Delete link object with id ``<x>`` |
-+-----------------------+-------------------------------------------+
++-------------------------+-------------------------------------------+
+| Argument | Description |
++=========================+===========================================+
+| ``-n``, ``--name <x>`` | Delete link object with name ``<x>`` |
++-------------------------+-------------------------------------------+
Example: ::
- delete link --lid 1
+ delete link --name linkName
Delete Job Function
@@ -422,15 +422,15 @@ Delete Job Function
Delete existing job object.
-+-----------------------+------------------------------------------+
-| Argument | Description |
-+=======================+==========================================+
-| ``-j``, ``--jid <x>`` | Delete job object with id ``<x>`` |
-+-----------------------+------------------------------------------+
++-------------------------+------------------------------------------+
+| Argument | Description |
++=========================+==========================================+
+| ``-n``, ``--name <x>`` | Delete job object with name ``<x>`` |
++-------------------------+------------------------------------------+
Example: ::
- delete job --jid 1
+ delete job --name jobName
Clone Command
@@ -443,15 +443,15 @@ Clone Link Function
Clone existing link object.
-+-----------------------+------------------------------------------+
-| Argument | Description |
-+=======================+==========================================+
-| ``-x``, ``--lid <x>`` | Clone link object with id ``<x>`` |
-+-----------------------+------------------------------------------+
++-------------------------+------------------------------------------+
+| Argument | Description |
++=========================+==========================================+
+| ``-n``, ``--name <x>`` | Clone link object with name ``<x>`` |
++-------------------------+------------------------------------------+
Example: ::
- clone link --lid 1
+ clone link --name linkName
Clone Job Function
@@ -459,15 +459,15 @@ Clone Job Function
Clone existing job object.
-+-----------------------+------------------------------------------+
-| Argument | Description |
-+=======================+==========================================+
-| ``-j``, ``--jid <x>`` | Clone job object with id ``<x>`` |
-+-----------------------+------------------------------------------+
++-------------------------+------------------------------------------+
+| Argument | Description |
++=========================+==========================================+
+| ``-n``, ``--name <x>`` | Clone job object with name ``<x>`` |
++-------------------------+------------------------------------------+
Example: ::
- clone job --jid 1
+ clone job --name jobName
Start Command
-------------
@@ -482,15 +482,15 @@ Start job (submit new submission). Starting already running job is considered as
+----------------------------+----------------------------+
| Argument | Description |
+============================+============================+
-| ``-j``, ``--jid <x>`` | Start job with id ``<x>`` |
+| ``-n``, ``--name <x>`` | Start job with name ``<x>``|
+----------------------------+----------------------------+
| ``-s``, ``--synchronous`` | Synchoronous job execution |
+----------------------------+----------------------------+
Example: ::
- start job --jid 1
- start job --jid 1 --synchronous
+ start job --name jobName
+ start job --name jobName --synchronous
Stop Command
------------
@@ -502,15 +502,15 @@ Stop Job Function
Interrupt running job.
-+-----------------------+------------------------------------------+
-| Argument | Description |
-+=======================+==========================================+
-| ``-j``, ``--jid <x>`` | Interrupt running job with id ``<x>`` |
-+-----------------------+------------------------------------------+
++-------------------------+------------------------------------------+
+| Argument | Description |
++=========================+==========================================+
+| ``-n``, ``--name <x>`` | Interrupt running job with name ``<x>`` |
++-------------------------+------------------------------------------+
Example: ::
- stop job --jid 1
+ stop job --name jobName
Status Command
--------------
@@ -522,12 +522,12 @@ Status Job Function
Retrieve last status for given job.
-+-----------------------+------------------------------------------+
-| Argument | Description |
-+=======================+==========================================+
-| ``-j``, ``--jid <x>`` | Retrieve status for job with id ``<x>`` |
-+-----------------------+------------------------------------------+
++-------------------------+------------------------------------------+
+| Argument | Description |
++=========================+==========================================+
+| ``-n``, ``--name <x>`` | Retrieve status for job with name ``<x>``|
++-------------------------+------------------------------------------+
Example: ::
- status job --jid 1
+ status job --name jobName
http://git-wip-us.apache.org/repos/asf/sqoop/blob/d89f64ad/docs/src/site/sphinx/user/Sqoop5MinutesDemo.rst
----------------------------------------------------------------------
diff --git a/docs/src/site/sphinx/user/Sqoop5MinutesDemo.rst b/docs/src/site/sphinx/user/Sqoop5MinutesDemo.rst
index d45f32e..0236f70 100644
--- a/docs/src/site/sphinx/user/Sqoop5MinutesDemo.rst
+++ b/docs/src/site/sphinx/user/Sqoop5MinutesDemo.rst
@@ -76,20 +76,20 @@ Creating Link Object
Check for the registered connectors on your Sqoop server: ::
sqoop:000> show connector
- +----+------------------------+----------------+------------------------------------------------------+----------------------+
- | Id | Name | Version | Class | Supported Directions |
- +----+------------------------+----------------+------------------------------------------------------+----------------------+
- | 1 | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
- | 2 | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
- +----+------------------------+----------------+------------------------------------------------------+----------------------+
+ +------------------------+----------------+------------------------------------------------------+----------------------+
+ | Name | Version | Class | Supported Directions |
+ +------------------------+----------------+------------------------------------------------------+----------------------+
+ | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
+ | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
+ +------------------------+----------------+------------------------------------------------------+----------------------+
-Our example contains two connectors. The one with connector Id 2 is called the ``generic-jdbc-connector``. This is a basic connector relying on the Java JDBC interface for communicating with data sources. It should work with the most common databases that are providing JDBC drivers. Please note that you must install JDBC drivers separately. They are not bundled in Sqoop due to incompatible licenses.
+Our example contains two connectors. The ``generic-jdbc-connector`` is a basic connector relying on the Java JDBC interface for communicating with data sources. It should work with the most common databases that are providing JDBC drivers. Please note that you must install JDBC drivers separately. They are not bundled in Sqoop due to incompatible licenses.
-Generic JDBC Connector in our example has a persistence Id 2 and we will use this value to create new link object for this connector. Note that the link name should be unique.
+Generic JDBC Connector in our example has a name ``generic-jdbc-connector`` and we will use this value to create new link object for this connector. Note that the link name should be unique.
::
- sqoop:000> create link -c 2
- Creating link for connector with id 2
+ sqoop:000> create link -connector generic-jdbc-connector
+ Creating link for connector with name generic-jdbc-connector
Please fill following values to create new link object
Name: First Link
@@ -101,22 +101,22 @@ Generic JDBC Connector in our example has a persistence Id 2 and we will use thi
JDBC Connection Properties:
There are currently 0 values in the map:
entry#protocol=tcp
- New link was successfully created with validation status OK and persistent id 1
+ New link was successfully created with validation status OK name First Link
-Our new link object was created with assigned id 1.
+Our new link object was created with assigned name First Link.
-In the ``show connector -all`` we see that there is a hdfs-connector registered in sqoop with the persistent id 1. Let us create another link object but this time for the hdfs-connector instead.
+In the ``show connector -all`` we see that there is a hdfs-connector registered. Let us create another link object but this time for the hdfs-connector instead.
::
- sqoop:000> create link -c 1
- Creating link for connector with id 1
+ sqoop:000> create link -connector hdfs-connector
+ Creating link for connector with name hdfs-connector
Please fill following values to create new link object
Name: Second Link
Link configuration
HDFS URI: hdfs://nameservice1:8020/
- New link was successfully created with validation status OK and persistent id 2
+ New link was successfully created with validation status OK and name Second Link
Creating Job Object
===================
@@ -127,8 +127,8 @@ Connectors implement the ``From`` for reading data from and/or ``To`` for writin
sqoop:000> show link --all
2 link(s) to show:
- link with id 1 and name First Link (Enabled: true, Created by root at 11/4/14 4:27 PM, Updated by root at 11/4/14 4:27 PM)
- Using Connector id 2
+ link with name First Link (Enabled: true, Created by root at 11/4/14 4:27 PM, Updated by root at 11/4/14 4:27 PM)
+ Using Connector with name generic-jdbc-connector
Link configuration
JDBC Driver Class: com.mysql.jdbc.Driver
JDBC Connection String: jdbc:mysql://mysql.ent.cloudera.com/sqoop
@@ -136,16 +136,16 @@ Connectors implement the ``From`` for reading data from and/or ``To`` for writin
Password:
JDBC Connection Properties:
protocol = tcp
- link with id 2 and name Second Link (Enabled: true, Created by root at 11/4/14 4:38 PM, Updated by root at 11/4/14 4:38 PM)
- Using Connector id 1
+ link with name Second Link (Enabled: true, Created by root at 11/4/14 4:38 PM, Updated by root at 11/4/14 4:38 PM)
+ Using Connector with name hdfs-connector
Link configuration
HDFS URI: hdfs://nameservice1:8020/
-Next, we can use the two link Ids to associate the ``From`` and ``To`` for the job.
+Next, we can use the two link names to associate the ``From`` and ``To`` for the job.
::
- sqoop:000> create job -f 1 -t 2
- Creating job for links with from id 1 and to id 2
+ sqoop:000> create job -f "First Link" -t "Second Link"
+ Creating job for links with from name First Link and to name Second Link
Please fill following values to create new job object
Name: Sqoopy
@@ -182,9 +182,9 @@ Next, we can use the two link Ids to associate the ``From`` and ``To`` for the j
Driver Config
Extractors:(Optional) 2
Loaders:(Optional) 2
- New job was successfully created with validation status OK and persistent id 1
+ New job was successfully created with validation status OK and name jobName
-Our new job object was created with assigned id 1. Note that if null value is allowed for the partition column,
+Our new job object was created with assigned name Sqoopy. Note that if null value is allowed for the partition column,
at least 2 extractors are needed for Sqoop to carry out the data transfer. On specifying 1 extractor in this
scenario, Sqoop shall ignore this setting and continue with 2 extractors.
@@ -194,9 +194,9 @@ Start Job ( a.k.a Data transfer )
You can start a sqoop job with the following command:
::
- sqoop:000> start job -j 1
+ sqoop:000> start job -name Sqoopy
Submission details
- Job ID: 1
+ Job Name: Sqoopy
Server URL: http://localhost:12000/sqoop/
Created by: root
Creation date: 2014-11-04 19:43:29 PST
@@ -209,9 +209,9 @@ You can iteratively check your running job status with ``status job`` command:
::
- sqoop:000> status job -j 1
+ sqoop:000> status job -n Sqoopy
Submission details
- Job ID: 1
+ Job Name: Sqoopy
Server URL: http://localhost:12000/sqoop/
Created by: root
Creation date: 2014-11-04 19:43:29 PST
@@ -224,9 +224,9 @@ Alternatively you can start a sqoop job and observe job running status with the
::
- sqoop:000> start job -j 1 -s
+ sqoop:000> start job -n Sqoopy -s
Submission details
- Job ID: 1
+ Job Name: Sqoopy
Server URL: http://localhost:12000/sqoop/
Created by: root
Creation date: 2014-11-04 19:43:29 PST
@@ -239,4 +239,4 @@ Alternatively you can start a sqoop job and observe job running status with the
And finally you can stop running the job at any time using ``stop job`` command: ::
- sqoop:000> stop job -j 1
+ sqoop:000> stop job -n Sqoopy