[jira] [Work logged] (BEAM-13242) IllegalArgumentException while reading a Numeric column with a fixed precision

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-13242?focusedWorklogId=693183&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-693183 ]

ASF GitHub Bot logged work on BEAM-13242:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 09/Dec/21 11:53
            Start Date: 09/Dec/21 11:53
    Worklog Time Spent: 10m 
      Work Description: Pavlmits commented on pull request #15987:
URL: https://github.com/apache/beam/pull/15987#issuecomment-989782555


   Hello people,
   
   I am facing the same issue. When this feature will be released? Thank you in advance.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 693183)
    Time Spent: 3.5h  (was: 3h 20m)

> IllegalArgumentException while reading a Numeric column with a fixed precision
> ------------------------------------------------------------------------------
>
>                 Key: BEAM-13242
>                 URL: https://issues.apache.org/jira/browse/BEAM-13242
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-jdbc
>    Affects Versions: 2.33.0
>         Environment: openjdk version "11.0.10" 2021-01-19
> macOS 11.1
>            Reporter: Denis
>            Assignee: Vitaly Ivanov
>            Priority: P2
>          Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> *Error*
>  
> I am trying to read a table in a Postgresql database. One of columns has a type definition 
> {code:java}
> Numeric(15, 2){code}
>  
> At runtime I am getting:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Expected BigDecimal base to be null or have precision = 15 (was 6), scale = 2 (was 2)	at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument(Preconditions.java:477)	at org.apache.beam.sdk.io.jdbc.LogicalTypes$FixedPrecisionNumeric.toInputType(LogicalTypes.java:268)	at org.apache.beam.sdk.io.jdbc.LogicalTypes$FixedPrecisionNumeric.toInputType(LogicalTypes.java:246)	at org.apache.beam.sdk.io.jdbc.SchemaUtil.lambda$createLogicalTypeExtractor$ca0ab2ec$1(SchemaUtil.java:289)	at org.apache.beam.sdk.io.jdbc.SchemaUtil$BeamRowMapper.mapRow(SchemaUtil.java:354)	at org.apache.beam.sdk.io.jdbc.SchemaUtil$BeamRowMapper.mapRow(SchemaUtil.java:332)	at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.processElement(JdbcIO.java:1172) {code}
>  
> It seems that when JdbcIO reads a numeric column it might have smaller precision/scale than defined in DDL.
>  
> *How to reproduce*
>  
> {code:java}
> import org.apache.beam.sdk.io.jdbc.JdbcIO
> import org.apache.beam.sdk.testing.PAssert
> import org.apache.beam.sdk.testing.TestPipeline
> import org.apache.beam.sdk.transforms.MapElements
> import org.apache.beam.sdk.transforms.ProcessFunction
> import org.apache.beam.sdk.values.TypeDescriptors
> import org.junit.AfterClass
> import org.junit.BeforeClass
> import org.junit.Test
> import org.testcontainers.containers.PostgreSQLContainer
> import java.math.BigDecimal
> import java.sql.Connection
> import java.sql.DriverManager
> class PostgresNumericBugTest {
>     companion object {
>         val pgContainer = PostgreSQLContainer<Nothing>("postgres:14.0")
>         @BeforeClass
>         @JvmStatic
>         fun start() {
>             pgContainer.start()
>             getConnection().use {
>                 val stmt = it.createStatement()
>                 stmt.execute(
>                     """create table account (
>                           id integer not null primary key,
>                           balance numeric(15, 2)
>                         )
>                     """.trimIndent()
>                 )
>                 stmt.execute("insert into account(id, balance) values(1, 5755.94)")
>             }
>         }
>         @AfterClass
>         @JvmStatic
>         fun stop() {
>             pgContainer.stop()
>         }
>         private fun getConnection(): Connection {
>             return DriverManager.getConnection(
>                 pgContainer.jdbcUrl, pgContainer.username, pgContainer.password
>             )
>         }
>         private fun getDataSourceConfiguration(): JdbcIO.DataSourceConfiguration =
>             JdbcIO.DataSourceConfiguration.create(
>                 pgContainer.driverClassName,
>                 pgContainer.jdbcUrl
>             )
>                 .withUsername(pgContainer.username)
>                 .withPassword(pgContainer.password)
>     }
>     @Test
>     fun readNumeric() {
>         val pipeline = TestPipeline.create().enableAbandonedNodeEnforcement(false)
>         val pBalance = pipeline
>             .apply(
>                 "Read Account from DB",
>                 JdbcIO
>                     .readRows()
>                     .withQuery("select balance from account")
>                     .withDataSourceConfiguration(getDataSourceConfiguration())
>             )
>             .apply(
>                 "Get Balance",
>                 MapElements.into(TypeDescriptors.bigdecimals()).via(ProcessFunction { it.getDecimal(0) })
>             )
>         PAssert.that(pBalance).containsInAnyOrder(BigDecimal("5755.94"))
>         pipeline.run()
>     }
> }
>  {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)