[jira] [Created] (BEAM-11906) No trigger early repeatedly for session windows

["Ning Kang (Jira)" <ji...@apache.org>]

Ning Kang created BEAM-11906:
--------------------------------

             Summary: No trigger early repeatedly for session windows
                 Key: BEAM-11906
                 URL: https://issues.apache.org/jira/browse/BEAM-11906
             Project: Beam
          Issue Type: Improvement
          Components: runner-dataflow
    Affects Versions: 2.28.0, 2.23.0
            Reporter: Ning Kang


Originated from: https://stackoverflow.com/questions/66381608/apache-beam-does-not-trigger-early-repeatedly-for-session-windows-on-google-data

The following pipeline fires early after each element when running locally using DirectRunner, but there are no early triggers when running on google cloud dataflow. On dataflow it triggers only after the session window has closed.

{code:python}
( p
        | 'read'   >> beam.io.ReadFromPubSub(subscription = 'projects/xxx/subscriptions/xxx-sub')
        | 'json'   >> beam.Map(lambda x: json.loads(x.decode('utf-8')))
        | 'kv'     >> beam.Map(lambda x: (x['id'], x['amount']))
        | 'window' >> beam.WindowInto(window.Sessions(15*60), trigger=trigger.Repeatedly(trigger.AfterCount(1)), accumulation_mode=AccumulationMode.ACCUMULATING)
        | 'group'  >> beam.GroupByKey()
        | 'log'    >> beam.Map(lambda x: logging.info(x))
)
{code}

Apache Beam versions tried: 2.23 and 2.28.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)