You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Sam Whittle (Jira)" <ji...@apache.org> on 2021/09/23 13:59:00 UTC

[jira] [Commented] (BEAM-12942) Dataflow runner specialization of PubsubIO should validate messages

    [ https://issues.apache.org/jira/browse/BEAM-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419231#comment-17419231 ] 

Sam Whittle commented on BEAM-12942:
------------------------------------

The validation seems like it could be helpful for other non-specialized PubsubIO because it would throw an exception when the message is produced instead of exception from pubsub publish result which could correspond to a batch and thus has less context.

> Dataflow runner specialization of PubsubIO should validate messages
> -------------------------------------------------------------------
>
>                 Key: BEAM-12942
>                 URL: https://issues.apache.org/jira/browse/BEAM-12942
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-dataflow
>            Reporter: Sam Whittle
>            Assignee: Sam Whittle
>            Priority: P2
>
> Currently invalid messages, such as those with attributes exceeding the maximum size are processed by the bundle successfully but fail to commit.
> Throwing an exception when trying to write such a message directly would increase visibility as well as allowing users to catch and handle such exceptions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)