[jira] [Commented] (BEAM-5654) Race condition in python sdk_worker causing KeyError

["Alex Amato (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638766#comment-16638766 ] 

Alex Amato commented on BEAM-5654:
----------------------------------

Note: I believe the code is able to recovery on the next iteration, I don't believe it causes pipelines to fail/get stuck. But does cause noisy errors to be logged.

> Race condition in python sdk_worker causing KeyError
> ----------------------------------------------------
>
>                 Key: BEAM-5654
>                 URL: https://issues.apache.org/jira/browse/BEAM-5654
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-py-core
>            Reporter: Alex Amato
>            Assignee: Ahmet Altay
>            Priority: Major
>
> We save Dataflow pipelines hitting a key error when hitting this line. This is likely due to a race condition between the key existence check and accessing it.
> [https://github.com/apache/beam/blob/v2.6.0/sdks/python/apache_beam/runners/worker/sdk_worker.py#L189]
> One fix would be to do a single lookup for the value instead of the double lookup.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)