[jira] [Created] (BEAM-10054) Direct Runner execution stalls with test pipeline

["Maximilian Michels (Jira)" <ji...@apache.org>]

Maximilian Michels created BEAM-10054:
-----------------------------------------

             Summary: Direct Runner execution stalls with test pipeline
                 Key: BEAM-10054
                 URL: https://issues.apache.org/jira/browse/BEAM-10054
             Project: Beam
          Issue Type: Bug
          Components: sdk-py-core
            Reporter: Maximilian Michels
            Assignee: Maximilian Michels


Internally, we have a test pipeline which runs with the DirectRunner. When upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:

{noformat}
tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb = None

    def raise_(tp, value=None, tb=None):
        """
        A function that matches the Python 2.x ``raise`` statement. This
        allows re-raising exceptions with the cls value and traceback on
        Python 2 and 3.
        """
        if value is not None and isinstance(tp, Exception):
            raise TypeError("instance exception may not have a separate value")
        if value is not None:
            exc = tp(value)
        else:
            exc = tp
        if exc.__traceback__ is not tb:
            raise exc.with_traceback(tb)
>       raise exc
E       Exception: Monitor task detected a pipeline stall.
{noformat}

I was able to bisect the error. This commit introduced the failure: https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731

If the following conditions evaluates to False, the pipeline runs correctly: https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273



--
This message was sent by Atlassian Jira
(v8.3.4#803005)