[jira] [Updated] (SAMZA-224) TestStatefulTask.testShouldStartAndRestore fails intermittently

["Martin Kleppmann (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martin Kleppmann updated SAMZA-224:
-----------------------------------

    Attachment: SAMZA-224.1.patch

Simplest possible fix: https://reviews.apache.org/r/22278/ (also attached as patch) — always start consuming from offset 0, whether or not the topic already exists when we start consuming.

With this change, we are no longer testing for SAMZA-142. I couldn't see a way of sensibly testing SAMZA-142 within the context of TestStatefulTask, without re-introducing the same race condition. I would much rather write unit tests for SamzaContainer (which contains the logic that treats changelog starting offset differently from input stream starting offset), rather than trying to cram everything into this very complex integration test.

Unfortunately, SamzaContainer in its current state is fairly difficult to write tests for. Its 360-line apply() method is not exactly test-friendly. I'd be keen to refactor it to make it more easily testable (and more readable), and write thorough tests at the same time. However, I don't think that needs to be a blocker for 0.7.0. Also, it wouldn't make much sense to start refactoring it before SAMZA-71 is committed, otherwise we'll just get even more merge conflicts.

So I propose committing just this small patch to master and 0.7.0 branches, and addressing the test for SAMZA-142 as part of a general SamzaContainer cleanup in 0.8.0. Does that sound reasonable?

> TestStatefulTask.testShouldStartAndRestore fails intermittently
> ---------------------------------------------------------------
>
>                 Key: SAMZA-224
>                 URL: https://issues.apache.org/jira/browse/SAMZA-224
>             Project: Samza
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.6.0
>            Reporter: Martin Kleppmann
>            Assignee: Martin Kleppmann
>             Fix For: 0.7.0
>
>         Attachments: SAMZA-224.1.patch
>
>
> [~closeuris] reported the following issue on SAMZA-185. I'm now intermittently seeing this problem too, so I'm opening a new issue for it (as I'm not sure the issue has anything to do with the Kafka 0.8.1 upgrade).
> {noformat}
> org.apache.samza.test.integration.TestStatefulTask > testShouldStartAndRestore FAILED
>     java.lang.AssertionError at TestStatefulTask.scala:356
> 2 tests completed, 1 failed
> :samza-test_2.10:test FAILED
> {noformat}
> Logs provided by Yan (mine look similar):
> Failed tests: http://pastebin.com/3nEDw9jC
> Standard output: http://pastebin.com/20vZbk7c
> Standard error: http://pastebin.com/gU6cTNAr



--
This message was sent by Atlassian JIRA
(v6.2#6252)