You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Juan Carlos Cardenas (JIRA)" <ji...@apache.org> on 2018/08/19 08:47:00 UTC
[jira] [Commented] (BEAM-3530) DoFn.process should raise exception
if something other than a List is returned
[ https://issues.apache.org/jira/browse/BEAM-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585072#comment-16585072 ]
Juan Carlos Cardenas commented on BEAM-3530:
--------------------------------------------
We added a validation, to check if the result is not iterable or if it is a string:
{code:java}
# validate results is iterable and non-string
try:
validate = iter(results)
except TypeError:
raise TypeError("This is not an iterable")
else:
if isinstance(results, str):
raise TypeError("This shouldn't be string")
{code}
> DoFn.process should raise exception if something other than a List is returned
> ------------------------------------------------------------------------------
>
> Key: BEAM-3530
> URL: https://issues.apache.org/jira/browse/BEAM-3530
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Chuan Yu Foo
> Priority: Major
> Labels: starter
>
> The process method of DoFns can either return values or yield values. In the case of returning values, it expects a List of elements to be returned. When returning a single value, it is easy to forget this, and return the value instead.
> Correct way:
> {{class SomeDoFn(beam.DoFn)}}
> {{ def process(self, elem):}}
> {{ return ['a']}}
> Incorrect way:
> {{class SomeDoFn(beam.DoFn)}}
> {{ def process(self, elem):}}
> {{ return 'a'}}
> A pipeline with the incorrect DoFn will fail will a cryptic error message without a direct indication that the actual error is due to SomeDoFn returning an element instead of a List containing that element. This issue is very time-consuming to track down.
> It would be good if the pipeline could raise an exception or otherwise indicate that the DoFn is incorrectly returning an element instead of a List to make it easier to identify the error.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)