You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Enis Soztutar (JIRA)" <ji...@apache.org> on 2009/11/12 09:55:39 UTC

[jira] Commented: (MAPREDUCE-1203) DBOutputFormat: add batch size support for JDBC and recieve DBWritable object in value not in key

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776912#action_12776912 ] 

Enis Soztutar commented on MAPREDUCE-1203:
------------------------------------------

The patch looks good, but I still have concerns about breaking the atomicity. 

If a map fails in the middle, then the DB will be filled with dangling records. In the second run, the driver will throw exception due to duplicate primary keys (at least for some DBs). Have you tested the patch with intentionally killing the task after commit some number of records?

> DBOutputFormat: add batch size support for JDBC and recieve  DBWritable object in value not in key
> --------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1203
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1203
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Alexander Schwid
>            Assignee: Aaron Kimball
>            Priority: Minor
>         Attachments: HADOOP-4331.patch, patch.txt
>
>
> package mapred.lib.db
> added batch size support for JDBC in DBOutputFormat 
> recieve  DBWritable object in value not in key in DBOutputFormat

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.