You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "nemon lou (JIRA)" <ji...@apache.org> on 2012/05/03 11:08:53 UTC

[jira] [Created] (SQOOP-482) import tool for mysql direct mode reports wrong record number

nemon lou created SQOOP-482:
-------------------------------

             Summary: import tool for mysql direct mode reports wrong record number
                 Key: SQOOP-482
                 URL: https://issues.apache.org/jira/browse/SQOOP-482
             Project: Sqoop
          Issue Type: Bug
          Components: connectors/mysql
    Affects Versions: 1.4.1-incubating
         Environment: hadoop 1.0.0
            Reporter: nemon lou
            Priority: Minor
             Fix For: 1.4.2-incubating


When run the sqoop import (via mysqldump) task :
sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
The console shows this(part of the log):
  12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
  12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268610#comment-13268610 ] 

jiraposter@reviews.apache.org commented on SQOOP-482:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4988/#review7580
-----------------------------------------------------------

Ship it!


Nemon's fix seems to be right to me. The write() method should be called only once per record. I also verified that his patch addresses the problem.

@Nemon
I am no commiter, so I cannot commit your patch. But you've got my vote! 

- Cheolsoo


On 2012-05-04 07:33:48, nemon lou wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4988/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-04 07:33:48)
bq.  
bq.  
bq.  Review request for Sqoop, Jarek Cecho and Cheolsoo Park.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  I simply change the way the MySQLDumpMapper writes contents.
bq.  May be there are other ways to fix it.Please let me know,thank you!
bq.  
bq.  
bq.  This addresses bug SQOOP-482.
bq.      https://issues.apache.org/jira/browse/SQOOP-482
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    ./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1332889 
bq.  
bq.  Diff: https://reviews.apache.org/r/4988/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  nemon
bq.  
bq.


                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267998#comment-13267998 ] 

jiraposter@reviews.apache.org commented on SQOOP-482:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4988/
-----------------------------------------------------------

(Updated 2012-05-04 01:02:58.521635)


Review request for Sqoop, Jarek Cecho and Cheolsoo Park.


Changes
-------

Add more review people!


Summary
-------

I simply change the way the MySQLDumpMapper writes contents.
May be there are other ways to fix it.Please let me know,thank you!


This addresses bug SQOOP-482.
    https://issues.apache.org/jira/browse/SQOOP-482


Diffs
-----

  ./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1332889 

Diff: https://reviews.apache.org/r/4988/diff


Testing
-------


Thanks,

nemon


                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269373#comment-13269373 ] 

Jarek Jarcec Cecho commented on SQOOP-482:
------------------------------------------

Arvind have to put Nemon to Sqoop contributor list before we will be able to assign ticket to him.

Jarcec
                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: SQOOP-482-1.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271951#comment-13271951 ] 

Hudson commented on SQOOP-482:
------------------------------

Integrated in Sqoop-ant-jdk-1.6 #112 (See [https://builds.apache.org/job/Sqoop-ant-jdk-1.6/112/])
    SQOOP-482. Import tool for mysql direct mode reports wrong
record number
(Nemon Lou via Kathleen Ting) (Revision 1336434)

     Result = SUCCESS
kathleen : 
Files : 
* /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java

                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Assignee: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: SQOOP-482-1.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268617#comment-13268617 ] 

jiraposter@reviews.apache.org commented on SQOOP-482:
-----------------------------------------------------



bq.  On 2012-05-04 18:36:37, Cheolsoo Park wrote:
bq.  > Nemon's fix seems to be right to me. The write() method should be called only once per record. I also verified that his patch addresses the problem.
bq.  > 
bq.  > @Nemon
bq.  > I am no commiter, so I cannot commit your patch. But you've got my vote!

@Kate
To be clear, I am not disagreeing with you. I am just talking about Nemon's diff file on reviewboard, and the patch on the jire has to be uploaded.

Hope that nobody misinterprets my comment. :-)


- Cheolsoo


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4988/#review7580
-----------------------------------------------------------


On 2012-05-04 07:33:48, nemon lou wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4988/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-04 07:33:48)
bq.  
bq.  
bq.  Review request for Sqoop, Jarek Cecho and Cheolsoo Park.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  I simply change the way the MySQLDumpMapper writes contents.
bq.  May be there are other ways to fix it.Please let me know,thank you!
bq.  
bq.  
bq.  This addresses bug SQOOP-482.
bq.      https://issues.apache.org/jira/browse/SQOOP-482
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    ./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1332889 
bq.  
bq.  Diff: https://reviews.apache.org/r/4988/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  nemon
bq.  
bq.


                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "Kathleen Ting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269065#comment-13269065 ] 

Kathleen Ting commented on SQOOP-482:
-------------------------------------

Nemon, please also assign the JIRA to yourself. Thanks.
                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: SQOOP-482-1.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268147#comment-13268147 ] 

jiraposter@reviews.apache.org commented on SQOOP-482:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4988/#review7546
-----------------------------------------------------------


Thank you very much for your patch. It looks good, but there are two minor coding convention issues. If you run "ant checkstyle", you will see the following errors that I commented inline. Would you mind fixing them and updating the patch?


./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java
<https://reviews.apache.org/r/4988/#comment16716>

    This line is longer than 80 chars.



./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java
<https://reviews.apache.org/r/4988/#comment16715>

    Please remove a tab char here.


- Cheolsoo


On 2012-05-04 01:02:58, nemon lou wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4988/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-04 01:02:58)
bq.  
bq.  
bq.  Review request for Sqoop, Jarek Cecho and Cheolsoo Park.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  I simply change the way the MySQLDumpMapper writes contents.
bq.  May be there are other ways to fix it.Please let me know,thank you!
bq.  
bq.  
bq.  This addresses bug SQOOP-482.
bq.      https://issues.apache.org/jira/browse/SQOOP-482
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    ./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1332889 
bq.  
bq.  Diff: https://reviews.apache.org/r/4988/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  nemon
bq.  
bq.


                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "nemon lou (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

nemon lou updated SQOOP-482:
----------------------------

    Attachment:     (was: directmodewrongnumber.patch)
    
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: SQOOP-482-1.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267986#comment-13267986 ] 

jiraposter@reviews.apache.org commented on SQOOP-482:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4988/#review7500
-----------------------------------------------------------


- nemon


On 2012-05-03 10:57:29, nemon lou wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4988/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-03 10:57:29)
bq.  
bq.  
bq.  Review request for Sqoop and Jarek Cecho.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  I simply change the way the MySQLDumpMapper writes contents.
bq.  May be there are other ways to fix it.Please let me know,thank you!
bq.  
bq.  
bq.  This addresses bug SQOOP-482.
bq.      https://issues.apache.org/jira/browse/SQOOP-482
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    ./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1332889 
bq.  
bq.  Diff: https://reviews.apache.org/r/4988/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  nemon
bq.  
bq.


                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "nemon lou (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

nemon lou updated SQOOP-482:
----------------------------

    Attachment: directmodewrongnumber.patch
    
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267325#comment-13267325 ] 

Jarek Jarcec Cecho commented on SQOOP-482:
------------------------------------------

Hi Nemon,
could you please upload your patch to Apache Review board (https://reviews.apache.org/) so that we can review it?

Jarcec
                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Issue Comment Edited] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "nemon lou (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268959#comment-13268959 ] 

nemon lou edited comment on SQOOP-482 at 5/5/12 1:17 PM:
---------------------------------------------------------

Update the JIRA patch
                
      was (Author: nemon):
    Update the JIRA path
                  
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: SQOOP-482-1.patch, directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "nemon lou (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267364#comment-13267364 ] 

nemon lou commented on SQOOP-482:
---------------------------------

I have upload it here https://reviews.apache.org/r/4988/diff/#index_header
                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268608#comment-13268608 ] 

jiraposter@reviews.apache.org commented on SQOOP-482:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4988/#review7581
-----------------------------------------------------------

Ship it!


+1

Nemon, thanks for the patch. Noticed that the timestamp of the patch uploaded to the jira is 5/3 but on review board the last diff was made on 5/4. Nemon, would you mind uploading the latest patch to the jira as SQOOP-482-1.patch?

- Kathleen


On 2012-05-04 07:33:48, nemon lou wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4988/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-04 07:33:48)
bq.  
bq.  
bq.  Review request for Sqoop, Jarek Cecho and Cheolsoo Park.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  I simply change the way the MySQLDumpMapper writes contents.
bq.  May be there are other ways to fix it.Please let me know,thank you!
bq.  
bq.  
bq.  This addresses bug SQOOP-482.
bq.      https://issues.apache.org/jira/browse/SQOOP-482
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    ./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1332889 
bq.  
bq.  Diff: https://reviews.apache.org/r/4988/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  nemon
bq.  
bq.


                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "Arvind Prabhakar (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arvind Prabhakar reassigned SQOOP-482:
--------------------------------------

    Assignee: nemon lou
    
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Assignee: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: SQOOP-482-1.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267363#comment-13267363 ] 

jiraposter@reviews.apache.org commented on SQOOP-482:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4988/
-----------------------------------------------------------

Review request for Sqoop and Jarek Cecho.


Summary
-------

I simply change the way the MySQLDumpMapper writes contents.
May be there are other ways to fix it.Please let me know,thank you!


This addresses bug SQOOP-482.
    https://issues.apache.org/jira/browse/SQOOP-482


Diffs
-----

  ./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1332889 

Diff: https://reviews.apache.org/r/4988/diff


Testing
-------


Thanks,

nemon


                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "nemon lou (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

nemon lou updated SQOOP-482:
----------------------------

    Attachment: SQOOP-482-1.patch

Update the JIRA path
                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: SQOOP-482-1.patch, directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "nemon lou (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269322#comment-13269322 ] 

nemon lou commented on SQOOP-482:
---------------------------------

Sorry.I could not find where to assign this issue to myself. :(
                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: SQOOP-482-1.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "Kathleen Ting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269907#comment-13269907 ] 

Kathleen Ting commented on SQOOP-482:
-------------------------------------

Thanks Arvind for adding Nemon to the list.
                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Assignee: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: SQOOP-482-1.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268190#comment-13268190 ] 

jiraposter@reviews.apache.org commented on SQOOP-482:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4988/
-----------------------------------------------------------

(Updated 2012-05-04 07:33:48.277299)


Review request for Sqoop, Jarek Cecho and Cheolsoo Park.


Changes
-------

checkstyle fixed


Summary
-------

I simply change the way the MySQLDumpMapper writes contents.
May be there are other ways to fix it.Please let me know,thank you!


This addresses bug SQOOP-482.
    https://issues.apache.org/jira/browse/SQOOP-482


Diffs (updated)
-----

  ./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1332889 

Diff: https://reviews.apache.org/r/4988/diff


Testing
-------


Thanks,

nemon


                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (SQOOP-482) import tool for mysql direct mode reports wrong record number

Posted by "nemon lou (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

nemon lou updated SQOOP-482:
----------------------------

    Comment: was deleted

(was: I have upload it here https://reviews.apache.org/r/4988/diff/#index_header)
    
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root -P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira