You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2017/03/30 16:15:20 UTC
[GitHub] flink pull request #3655: [FLINK-6175] Harden HistoryServerTest#testFullArch...
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/3655
[FLINK-6175] Harden HistoryServerTest#testFullArchiveLifecycle
This PR hardens the `testFullArchiveLifecycle` test by introducing an optional `CountdownLatch` argument to the HistoryServer constructor This latch is updated after each finished archive poll, regardless of whether it failed or not.
This simplifies the test and removes edge cases where the requested file already exists but the contents weren't written yet.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 6175_hs_test
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3655.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3655
----
commit 3e491a532330476ecf05f2eb319a95212569b52d
Author: zentol <ch...@apache.org>
Date: 2017-03-23T12:58:03Z
[FLINK-6175] Harden HistoryServerTest#testFullArchiveLifecycle
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3655: [FLINK-6175] Harden HistoryServerTest#testFullArchiveLife...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/3655
While at hardening: The test seems to work with a fix port. That is bound to cause failures due to conflicts (for example 8082 is common to be used if you have a local HA Flink for testing).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3655: [FLINK-6175] Harden HistoryServerTest#testFullArchiveLife...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/3655
@StephanEwen Good idea, updated the PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3655: [FLINK-6175] Harden HistoryServerTest#testFullArchiveLife...
Posted by haohui <gi...@git.apache.org>.
Github user haohui commented on the issue:
https://github.com/apache/flink/pull/3655
Looks good to me overall. Instead of changing in the normal code path, maybe it is a little bit less intrusive to use Mockito to change the field? For example:
```
JobArchiveFetcherTask task = Whitebox.getInternal(archiveFetcher, "fetcherTask");
JobArchiveFetcherTask spiedTask = Mockito.spy(task);
Whitebox.setInternal(archiveFetcher, "fetcherTask", spiedTask);
doAnswer(answer -> latch.countDown()).doCallRealMethod().when(spiedTask).run();
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3655: [FLINK-6175] Harden HistoryServerTest#testFullArchiveLife...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/3655
hmm...this didn't work as well as I'd hoped; its still failing on travis. Will investigate a bit more.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3655: [FLINK-6175] Harden HistoryServerTest#testFullArchiveLife...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/3655
It would be less intrusive, that's true. I do prefer staying away from mockito magic as far as possible though Mockito often results in failing tests even though no public contract was modified/violated: in those 4 lines alone you rely on 3 _internal_ details to not change for the test to succeed (name of the Field, class of the Field, way of execution (run()).
This now means that when the implementation changes so must the test, which kind of defeats their purpose of comparing a new implementation to a known use-case; as well as introducing a risk for additional bugs to sneak in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3655: [FLINK-6175] Harden HistoryServerTest#testFullArch...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/3655
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3655: [FLINK-6175] Harden HistoryServerTest#testFullArchiveLife...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/3655
merging.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---