[jira] [Work logged] (BEAM-6934) Unexpected TestStream behavior when testing stateful DoFn (DirectRunner)

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-6934?focusedWorklogId=224632&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-224632 ]

ASF GitHub Bot logged work on BEAM-6934:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 08/Apr/19 22:12
            Start Date: 08/Apr/19 22:12
    Worklog Time Spent: 10m 
      Work Description: pabloem commented on pull request #8252: [BEAM-6934] Fixing timer firing timing issue
URL: https://github.com/apache/beam/pull/8252
 
 
   What is odd about this problem is that:
   - We would expect the timer set to `100` to fire when the watermark advances to 100 - but instead, it fires together with the timer set to `1000` when the watermark advances to infinity.
   
   On the other hand, if we set `advance_watermark_to(101)`, the test behaves properly without any need for modifications.
   
   BUT, what's even more odd is that when doing the actual check in trigger.py as to whether the timer should be fired, we do check for `<=`. Strange.:
   https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/trigger.py#L1273
   
   Nonetheless, this change does fix the new test (which was previously failing) - and I believe it does correct the behavior. So. Yes.
   
   I'll keep hunting for bugs.
   
   r: @aaltay 
   cc: @tweise 
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 224632)
            Time Spent: 10m
    Remaining Estimate: 0h

> Unexpected TestStream behavior when testing stateful DoFn (DirectRunner)
> ------------------------------------------------------------------------
>
>                 Key: BEAM-6934
>                 URL: https://issues.apache.org/jira/browse/BEAM-6934
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>            Reporter: Ahmet Altay
>            Assignee: Pablo Estrada
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> User reported issue:
> """
> I'm running into some unexpected behavior when trying to unit test a stateful DoFn that uses watermark timers as well as bag state.  I'm following the example here: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/userstate_test.py#L333
> Expected behavior:
> When using TestStream, if a stateful DoFn adds value 'foo' to BagState then sets two watermark timers t1 and t2, where t1 clears the bag state and t2 reads from bag state and emits the contents, if t1.time < t2.time then nothing should get emitted when the TestPipeline is run. (bag state should be cleared by timer at t1 before it is read from by timer at t2)
> Actual behavior:
> For the scenario described above, results get emitted despite t1.time being less than t2.time.
> I've created a gist with a demonstration of the problem:
> https://gist.github.com/jcruelty/3bf5ce5865110372a2d1650b1421cde1
> """
> (cc: [~ccy])



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)