You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2022/03/13 16:59:00 UTC
[jira] [Updated] (BEAM-13905) Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.
[ https://issues.apache.org/jira/browse/BEAM-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Beam JIRA Bot updated BEAM-13905:
---------------------------------
Labels: stale-assigned (was: )
> Apache Beam Python: Dataframe Transforms break when the option runtime_type_check is enabled.
> ---------------------------------------------------------------------------------------------
>
> Key: BEAM-13905
> URL: https://issues.apache.org/jira/browse/BEAM-13905
> Project: Beam
> Issue Type: Bug
> Components: dsl-dataframe
> Affects Versions: 2.35.0
> Environment: OS: Linux
> Python 3.8.12
> Reporter: Benoit Clennett-Sirois
> Assignee: Brian Hulette
> Priority: P2
> Labels: stale-assigned
> Fix For: 2.38.0
>
>
> We have discovered a potential bug whereas when you execute a pipeline that contains
> a DataframeTransform with the "runtime_type_check" option set to True, a cryptic
> error is raised by Apache Beam typecheckng.
> Simple example to reproduce the bug:
>
> {code:java}
> from apache_beam.options.pipeline_options import PipelineOptions
> from apache_beam import Pipeline, Create, Row
> from apache_beam.dataframe.transforms import DataframeTransform
> pipeline = Pipeline(options=PipelineOptions(runtime_type_check=True))
> pipeline | Create([Row(val1=1)]) | DataframeTransform(lambda df: df)
> pipeline.run(){code}
> This raises a apache_beam.typehints.decorators.TypeCheckError:
> {code:java}
> File ".....lib/python3.8/site-packages/apache_beam/typehints/typehints.py", line 416, in check_constraint
> raise SimpleTypeHintError
> apache_beam.typehints.decorators.TypeCheckError: According to type-hint expected output should be of type <class 'apache_beam.typehints.schemas.BeamSchema_118086df_671f_4643_a929_ba65de48e7e8'>. Instead, received 'BeamSchema_118086df_671f_4643_a929_ba65de48e7e8(val1=1)', an instance of type <class 'apache_beam.typehints.schemas.BeamSchema_118086df_671f_4643_a929_ba65de48e7e8'>. [while running 'DataframeTransform/Unbatch 'placeholder_DataFrame_140623617251840'/ParDo(_UnbatchNoIndex)'] {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)