[jira] [Created] (BEAM-3500) JdbcIO: Improve connection management

["Pawel Bartoszek (JIRA)" <ji...@apache.org>]

Pawel Bartoszek created BEAM-3500:
-------------------------------------

             Summary: JdbcIO: Improve connection management
                 Key: BEAM-3500
                 URL: https://issues.apache.org/jira/browse/BEAM-3500
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-extensions
    Affects Versions: 2.2.0
            Reporter: Pawel Bartoszek
            Assignee: Reuven Lax


JdbcIO write DoFn acquires connection in {{@Setup}} and release it in {{@Teardown}} methods, which means that connection might stay opened for days in streaming job case. Keeping single connection open for so long might be very risky as it's exposed to database, network etc issues.

*Taking connection from the pool when it is actually needed*

I suggest that connection would be taken from the connection pool in {{executeBatch}} method and released when the batch is flushed. This will allow the pool to take care of any returned unhealthy connections etc.

*Make JdbcIO accept data source factory*
 It would be nice if JdbcIO accepted DataSourceFactory rather than DataSource itself. I am saying that because sink checks if DataSource implements `Serializable` interface, which make it impossible to pass BasicDataSource(used internally by sink) as it doesn’t implement this interface. Something like:
{code:java}
interface DataSourceFactory extends Serializable{
     DataSource create();
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)