You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Kannan Rajah (JIRA)" <ji...@apache.org> on 2015/01/31 01:12:38 UTC

[jira] [Updated] (MAPREDUCE-6237) DBRecordReader is not thread safe

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kannan Rajah updated MAPREDUCE-6237:
------------------------------------
    Description: 
DBInputFormat.createDBRecorder is reusing JDBC connections across instances of DBRecordReader. This is not a good idea. We should be creating separate connection. If performance is a concern, then we should be using connection pooling instead.

I looked at DBOutputFormat.getRecordReader. It actually creates a new Connection object for each DBRecordReader. So can we just change DBInputFormat to create new Connection every time? The connection reuse code was added as part of connection leak bug in MAPREDUCE-1443. Any reason for caching the connection?

  was:
DBInputFormat.createDBRecorder is reusing JDBC connections across
instances of DBRecordReader. This is not a good idea. We should be creating separate connection. If performance is a concern, then we should be using connection pooling instead.

I looked at DBOutputFormat.getRecordReader. It actually creates a
new Connection object for each DBRecordReader. So can we just change
DBInputFormat to create new Connection every time?

The connection reuse code was added as part of connection leak bug in 
MAPREDUCE-1443. Any reason for caching the connection?


> DBRecordReader is not thread safe
> ---------------------------------
>
>                 Key: MAPREDUCE-6237
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6237
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.5.0
>            Reporter: Kannan Rajah
>            Assignee: Kannan Rajah
>
> DBInputFormat.createDBRecorder is reusing JDBC connections across instances of DBRecordReader. This is not a good idea. We should be creating separate connection. If performance is a concern, then we should be using connection pooling instead.
> I looked at DBOutputFormat.getRecordReader. It actually creates a new Connection object for each DBRecordReader. So can we just change DBInputFormat to create new Connection every time? The connection reuse code was added as part of connection leak bug in MAPREDUCE-1443. Any reason for caching the connection?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)