You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Veena Basavaraj (JIRA)" <ji...@apache.org> on 2014/11/13 01:15:33 UTC
[jira] [Commented] (SQOOP-1717) Sqoop2: Remove Data class from docs
[ https://issues.apache.org/jira/browse/SQOOP-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208965#comment-14208965 ]
Veena Basavaraj commented on SQOOP-1717:
----------------------------------------
is this clean up of data class going to the 1.99.4?
> Sqoop2: Remove Data class from docs
> -----------------------------------
>
> Key: SQOOP-1717
> URL: https://issues.apache.org/jira/browse/SQOOP-1717
> Project: Sqoop
> Issue Type: Bug
> Components: docs
> Affects Versions: 1.99.4
> Reporter: Abraham Elmahrek
> Assignee: Veena Basavaraj
>
> From ConnectorDevelopment.java:
> {noformat}
> The diagram below describes the initialization phase of a job.
> ``SqoopInputFormat`` create splits using ``Partitioner``.
> ::
> ,----------------. ,-----------.
> |SqoopInputFormat| |Partitioner|
> `-------+--------' `-----+-----'
> getSplits | |
> ----------->| |
> | getPartitions |
> |------------------------>|
> | | ,---------.
> | |-------> |Partition|
> | | `----+----'
> |<- - - - - - - - - - - - | |
> | | | ,----------.
> |-------------------------------------------------->|SqoopSplit|
> | | | `----+-----'
> The diagram below describes the map phase of a job.
> ``SqoopMapper`` invokes ``From`` connector's extractor's ``extract`` method.
> ::
> ,-----------.
> |SqoopMapper|
> `-----+-----'
> run |
> --------->| ,------------------.
> |---------------------------------->|SqoopMapDataWriter|
> | `------+-----------'
> | ,---------. |
> |--------------> |Extractor| |
> | `----+----' |
> | extract | |
> |-------------------->| |
> | | |
> read from DB | |
> <-------------------------------| write* |
> | |------------------->|
> | | | ,----.
> | | |---------->|Data|
> | | | `-+--'
> | | |
> | | | context.write
> | | |-------------------------->
> The diagram below decribes the reduce phase of a job.
> ``OutputFormat`` invokes ``To`` connector's loader's ``load`` method (via ``SqoopOutputFormatLoadExecutor`` ).
> ::
> ,------------. ,---------------------.
> |SqoopReducer| |SqoopNullOutputFormat|
> `---+--------' `----------+----------'
> | | ,-----------------------------.
> | |-> |SqoopOutputFormatLoadExecutor|
> | | `--------------+--------------' ,----.
> | | |---------------------> |Data|
> | | | `-+--'
> | | | ,-----------------. |
> | | |-> |SqoopRecordWriter| |
> getRecordWriter | | `--------+--------' |
> ----------------------->| getRecordWriter | | |
> | |----------------->| | | ,--------------.
> | | |-----------------------------> |ConsumerThread|
> | | | | | `------+-------'
> | |<- - - - - - - - -| | | | ,------.
> <- - - - - - - - - - - -| | | | |--->|Loader|
> | | | | | | `--+---'
> | | | | | | |
> | | | | | | load |
> run | | | | | |------>|
> ----->| | write | | | | |
> |------------------------------------------------>| setContent | | read* |
> | | | |----------->| getContent |<------|
> | | | | |<-----------| |
> | | | | | | - - ->|
> | | | | | | | write into DB
> | | | | | | |-------------->
> .. _`Intermediate representation`: https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Intermediate+representation
> {noformat}
> Notice the use of the "Data" entity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)