[jira] [Updated] (BEAM-5305) Timeout handling in JdbcIO with Oracle java driver

["Kenneth Knowles (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Knowles updated BEAM-5305:
----------------------------------
    Priority: P2  (was: P1)

> Timeout handling in JdbcIO with Oracle java driver
> --------------------------------------------------
>
>                 Key: BEAM-5305
>                 URL: https://issues.apache.org/jira/browse/BEAM-5305
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-jdbc
>    Affects Versions: 2.5.0
>            Reporter: Byambasuren Ganbaatar
>            Priority: P2
>              Labels: JdbcIO, oracle
>
> Hello, we have a batch pipeline that is reading data from Oracle, writing it to BigQuery.
> Using JdbcIO, BigQueryIO and Oracle driver [3].
> We run this job 2 times in a day, the pipeline usually works fine, it completed within 10 minutes.
> But sometimes we have a [1] warning from this job, it never finished until we cancelling it.
> (sometimes means 4 times in last month)
> Even we set timeout configuration for our code [2], warning log says [1] Processing stuck in step Read JDBC.
>  
> Would you explain and correct us this kind of problem? 
> We already contacted Google Support Team, they said our configuration code is looked fine them. Then  they instructed us to ask here.
> Thanks,
> Bya
> [1] Warning Log
> {code:java}
> Processing stuck in step Read JDBC/JdbcIO.Read/JdbcIO.ReadAll/ParDo(Read) for at least 13h40m00s without outputting or completing in state process at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:197) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at oracle.net.nt.TimeoutSocketChannel.read(TimeoutSocketChannel.java:144) at oracle.net.ns.NIOHeader.readHeaderBuffer(NIOHeader.java:82) at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:139) at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:101) at oracle.net.ns.NIONSDataChannel.readDataFromSocketChannel(NIONSDataChannel.java:80) at oracle.jdbc.driver.T4CMAREngineNIO.prepareForReading(T4CMAREngineNIO.java:98) at oracle.jdbc.driver.T4CMAREngineNIO.unmarshalUB1(T4CMAREngineNIO.java:534) at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:485) at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252) at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612) at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:226) at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:59) at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:747) at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:904) at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1082) at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3780) at oracle.jdbc.driver.T4CPreparedStatement.executeInternal(T4CPreparedStatement.java:1343) at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3822) at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1165) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83) at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.processElement(JdbcIO.java:601) at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn$DoFnInvoker.invokeProcessElement(Unknown Source)
> {code}
>  
> [2] Jdbc configure
> {code:java}
> import org.apache.beam.sdk.io.jdbc.JdbcIO;
> import oracle.jdbc.driver.OracleConnection;
> JdbcIO.DataSourceConfiguration  dataSource = JdbcIO
>                 .DataSourceConfiguration.create("oracle.jdbc.driver.OracleDriver", oracleUrl)
>                 .withUsername(oracleUser)
>                 .withPassword(oraclePassword)
>                 .withConnectionProperties(String.format("%s=%s",
>                         OracleConnection.CONNECTION_PROPERTY_THIN_NET_CONNECT_TIMEOUT, 10000))
>                 .withConnectionProperties(String.format("%s=%s",
>                         OracleConnection.CONNECTION_PROPERTY_THIN_READ_TIMEOUT, 600000))
>                 .withConnectionProperties(String.format("%s=%s",
>                         OracleConnection.CONNECTION_PROPERTY_TIMEZONE_AS_REGION, false))
>                 .withConnectionProperties(String.format("%s=%s",
>                         OracleConnection.CONNECTION_PROPERTY_NET_KEEPALIVE, false))
>         ;
> pipeline.apply(JdbcIO.<TableRow>read()
>     .withDataSourceConfiguration(dataSource)
>     .withQuery(readQuery)
>     .withCoder(TableRowJsonCoder.of())
>     .withRowMapper(new TableRowRowMapper())
> );
> {code}
>  
> [3] Oracle DB driver: ojdbc8 12.2.0.1
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)