You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Veena Basavaraj (JIRA)" <ji...@apache.org> on 2014/11/18 17:37:33 UTC
[jira] [Created] (SQOOP-1753) Sqoop2: CommandLineClient docs have
formatting issues
Veena Basavaraj created SQOOP-1753:
--------------------------------------
Summary: Sqoop2: CommandLineClient docs have formatting issues
Key: SQOOP-1753
URL: https://issues.apache.org/jira/browse/SQOOP-1753
Project: Sqoop
Issue Type: Bug
Components: docs
Affects Versions: 1.99.5
Reporter: Veena Basavaraj
Assignee: Veena Basavaraj
Fix For: 1.99.5
>From ConnectorDevelopment.java:
{noformat}
The diagram below describes the initialization phase of a job.
``SqoopInputFormat`` create splits using ``Partitioner``.
::
,----------------. ,-----------.
|SqoopInputFormat| |Partitioner|
`-------+--------' `-----+-----'
getSplits | |
----------->| |
| getPartitions |
|------------------------>|
| | ,---------.
| |-------> |Partition|
| | `----+----'
|<- - - - - - - - - - - - | |
| | | ,----------.
|-------------------------------------------------->|SqoopSplit|
| | | `----+-----'
The diagram below describes the map phase of a job.
``SqoopMapper`` invokes ``From`` connector's extractor's ``extract`` method.
::
,-----------.
|SqoopMapper|
`-----+-----'
run |
--------->| ,------------------.
|---------------------------------->|SqoopMapDataWriter|
| `------+-----------'
| ,---------. |
|--------------> |Extractor| |
| `----+----' |
| extract | |
|-------------------->| |
| | |
read from DB | |
<-------------------------------| write* |
| |------------------->|
| | | ,----.
| | |---------->|Data|
| | | `-+--'
| | |
| | | context.write
| | |-------------------------->
The diagram below decribes the reduce phase of a job.
``OutputFormat`` invokes ``To`` connector's loader's ``load`` method (via ``SqoopOutputFormatLoadExecutor`` ).
::
,------------. ,---------------------.
|SqoopReducer| |SqoopNullOutputFormat|
`---+--------' `----------+----------'
| | ,-----------------------------.
| |-> |SqoopOutputFormatLoadExecutor|
| | `--------------+--------------' ,----.
| | |---------------------> |Data|
| | | `-+--'
| | | ,-----------------. |
| | |-> |SqoopRecordWriter| |
getRecordWriter | | `--------+--------' |
----------------------->| getRecordWriter | | |
| |----------------->| | | ,--------------.
| | |-----------------------------> |ConsumerThread|
| | | | | `------+-------'
| |<- - - - - - - - -| | | | ,------.
<- - - - - - - - - - - -| | | | |--->|Loader|
| | | | | | `--+---'
| | | | | | |
| | | | | | load |
run | | | | | |------>|
----->| | write | | | | |
|------------------------------------------------>| setContent | | read* |
| | | |----------->| getContent |<------|
| | | | |<-----------| |
| | | | | | - - ->|
| | | | | | | write into DB
| | | | | | |-------------->
.. _`Intermediate representation`: https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Intermediate+representation
{noformat}
Notice the use of the "Data" entity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)