[GitHub] [beam] damccorm opened a new issue, #21428: Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.

[GitBox <gi...@apache.org>]

damccorm opened a new issue, #21428:
URL: https://github.com/apache/beam/issues/21428

   We have discovered a potential bug whereas when you execute a pipeline that contains
   a DataframeTransform with the "runtime_type_check" option set to True, a cryptic
   error is raised by Apache Beam typecheckng.
   
   Simple example to reproduce the bug:
       
   ```
   
   from apache_beam.options.pipeline_options import PipelineOptions
   from apache_beam import Pipeline,
   Create, Row
   from apache_beam.dataframe.transforms import DataframeTransform
   pipeline = Pipeline(options=PipelineOptions(runtime_type_check=True))
   pipeline
   | Create([Row(val1=1)]) | DataframeTransform(lambda df: df)
   pipeline.run()
   ```
   
   This raises a apache_beam.typehints.decorators.TypeCheckError:
   ```
   
   File ".....lib/python3.8/site-packages/apache_beam/typehints/typehints.py", line 416, in check_constraint
    
     raise SimpleTypeHintError
   apache_beam.typehints.decorators.TypeCheckError: According to type-hint
   expected output should be of type <class 'apache_beam.typehints.schemas.BeamSchema_118086df_671f_4643_a929_ba65de48e7e8'>.
   Instead, received 'BeamSchema_118086df_671f_4643_a929_ba65de48e7e8(val1=1)', an instance of type <class
   'apache_beam.typehints.schemas.BeamSchema_118086df_671f_4643_a929_ba65de48e7e8'>. [while running 'DataframeTransform/Unbatch
   'placeholder_DataFrame_140623617251840'/ParDo(_UnbatchNoIndex)'] 
   ```
   
    
   
   Imported from Jira [BEAM-13905](https://issues.apache.org/jira/browse/BEAM-13905). Original Jira may contain additional context.
   Reported by: benwah.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] damccorm commented on issue #21428: Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.

[GitBox <gi...@apache.org>]

damccorm commented on issue #21428:
URL: https://github.com/apache/beam/issues/21428#issuecomment-1146702100

   Unable to assign user @theneuralbit. If able, self-assign, otherwise tag @damccorm so that he can assign you. Because of GitHub's spam prevention system, your activity is required to enable assignment in this repo.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org