[I] [Bug]: Python JDBCIO failed to handle DATE fields [beam]

["shunping (via GitHub)" <gi...@apache.org>]

shunping opened a new issue, #35244:
URL: https://github.com/apache/beam/issues/35244

   ### What happened?
   
   Steps to reproduce:
   1. Create a table in a database (e.g. Postgres) and insert rows
     ```SQL
      CREATE TABLE my_test (ITEM VARCHAR(25), LAST_TS DATE);
      INSERT INTO my_test (ITEM, LAST_TS) VALUES ('item1', '2025-06-15');
      INSERT INTO my_test (ITEM, LAST_TS) VALUES ('item2', '2025-06-16');
      INSERT INTO my_test (ITEM, LAST_TS) VALUES ('item3', NULL);
     ```
   2. Run the following python code
   ```python
   import logging
   import apache_beam as beam
   from apache_beam.options.pipeline_options import PipelineOptions
   from apache_beam.io.jdbc import ReadFromJdbc
   
   logging.basicConfig(level=logging.INFO)
   
   options = PipelineOptions()
   
   driver = 'org.postgresql.Driver'
   jdbc_url = 'jdbc:postgresql://localhost:5432/<your_db>'
   username = 'postgres'
   password = ''
   
   
   with beam.Pipeline(options=options) as p:
     _ = (
         p
         | 'Read from jdbc' >> ReadFromJdbc(
             table_name="my_test",
             driver_class_name=driver,
             jdbc_url=jdbc_url,
             username=username,
             password=password)
         )
         | beam.Map(print))
   
   ```
   
   The following error is raised when running the pipeline.
   ```
     File "../sdks/python/apache_beam/typehints/schemas.py", line 926, in to_language_type
       return Timestamp(seconds=int(value.seconds), micros=int(value.micros))
                                ^^^^^^^^^^^^^^^^^^
   TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
   ```
   
   
   There is a workaround to put the following lines before the "with" statement.
   ```
   from apache_beam.coders.row_coder import LogicalType
   from apache_beam.typehints.schemas import MillisInstant
   LogicalType.register_logical_type(MillisInstant)
   ```
   ```
   
   ### Issue Priority
   
   Priority: 2 (default / most bugs should be filed as P2)
   
   ### Issue Components
   
   - [ ] Component: Python SDK
   - [ ] Component: Java SDK
   - [ ] Component: Go SDK
   - [ ] Component: Typescript SDK
   - [ ] Component: IO connector
   - [ ] Component: Beam YAML
   - [ ] Component: Beam examples
   - [ ] Component: Beam playground
   - [ ] Component: Beam katas
   - [ ] Component: Website
   - [ ] Component: Infrastructure
   - [ ] Component: Spark Runner
   - [ ] Component: Flink Runner
   - [ ] Component: Samza Runner
   - [ ] Component: Twister2 Runner
   - [ ] Component: Hazelcast Jet Runner
   - [ ] Component: Google Cloud Dataflow Runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug]: Python JDBCIO failed to handle DATE fields [beam]

["Abacn (via GitHub)" <gi...@apache.org>]

Abacn commented on issue #35244:
URL: https://github.com/apache/beam/issues/35244#issuecomment-2963959779

   This is a known issue, a workaround is to add this line before pipeline construction, as done by test
   
   https://github.com/apache/beam/blob/7f09169252a77dcef024104a7ee9fb5c2717c4bc/sdks/python/apache_beam/io/external/xlang_jdbcio_it_test.py#L260


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug]: Python JDBCIO failed to handle DATE fields [beam]

["shunping (via GitHub)" <gi...@apache.org>]

shunping closed issue #35244: [Bug]: Python JDBCIO failed to handle DATE fields
URL: https://github.com/apache/beam/issues/35244


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org