You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zhangminglei <gi...@git.apache.org> on 2018/04/22 07:23:16 UTC
[GitHub] flink pull request #5890: [FLINK-8999] [e2e] Ensure the job has an operator ...
GitHub user zhangminglei opened a pull request:
https://github.com/apache/flink/pull/5890
[FLINK-8999] [e2e] Ensure the job has an operator with operator state
## What is the purpose of the change
This commit that creates a job which contains an operator that use operator state. By using a map function using that state.
## Brief change log
Add a test job that ensure a job contains an operator state.
Add a test script that verify the change.
## Verifying this change
Use the added script verify the change, or use run-pre-commit-tests.sh to trigger the test by Travis CI.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? ( not documented)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zhangminglei/flink flink-8999-managed_operator_state
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5890.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5890
----
commit 6189aeffdbd1e179c2ffe0c9d0873898c2a26f3a
Author: zhangminglei <zm...@...>
Date: 2018-04-22T07:14:57Z
[FLINK-8999] [e2e] Ensure the job has an operator with operator state
----
---
[GitHub] flink issue #5890: [FLINK-8999] [e2e] Ensure the job has an operator with op...
Posted by StefanRRichter <gi...@git.apache.org>.
Github user StefanRRichter commented on the issue:
https://github.com/apache/flink/pull/5890
Hi @zhangminglei . I think there might unfortunately be a small misunderstanding: this issue was a subtask for the task of creating a single general purpose test job, i.e. a single job that covers as many features of Flink as possible for testing. Of course, that includes adding an operator with operator state to that particular job. The goal here was not to have separate jobs for all of the tasks in the list, but have them all in one job, so that we can also be sure that they integrate well with each other.
Maybe @tzulitai can take a look if we can use some part of your code and integrate it with the job.
---
[GitHub] flink issue #5890: [FLINK-8999] [e2e] Ensure the job has an operator with op...
Posted by tzulitai <gi...@git.apache.org>.
Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/5890
Hi @zhangminglei, I think the current general purpose DataStream job already subsumes this PR; for example, the sequence source generator in the job already uses operator state. Moreover, `test_resume_savepoint.sh` also tests the behaviours of that via a rescale operation.
What do you think? If you agree, we maybe can close this PR.
---
[GitHub] flink issue #5890: [FLINK-8999] [e2e] Ensure the job has an operator with op...
Posted by zhangminglei <gi...@git.apache.org>.
Github user zhangminglei commented on the issue:
https://github.com/apache/flink/pull/5890
Hello, @StefanRRichter . Could you review this code please ?
---
[GitHub] flink issue #5890: [FLINK-8999] [e2e] Ensure the job has an operator with op...
Posted by zhangminglei <gi...@git.apache.org>.
Github user zhangminglei commented on the issue:
https://github.com/apache/flink/pull/5890
@tzulitai Yes. Thanks ~ . I totally agree. I have been so busy these days. I will close this PR.
---
[GitHub] flink pull request #5890: [FLINK-8999] [e2e] Ensure the job has an operator ...
Posted by zhangminglei <gi...@git.apache.org>.
Github user zhangminglei closed the pull request at:
https://github.com/apache/flink/pull/5890
---