You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by wuchong <gi...@git.apache.org> on 2016/08/11 08:17:10 UTC
[GitHub] flink pull request #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowed...
GitHub user wuchong opened a pull request:
https://github.com/apache/flink/pull/2354
[FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful description of your changes.
- [x] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [x] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis build has passed
This PR avoid users try to set a parallelism on an all-windowed stream.
I add an `isAllWindow` field to mark a transformation is all-windowed. I'm appreciate if there is better solutions.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wuchong/flink FLINK-4366
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2354.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2354
----
commit 79ac99a715ef9bf15de0e5f0cd8080ebc8d278c6
Author: Jark Wu <wu...@alibaba-inc.com>
Date: 2016-08-11T08:04:54Z
[FLINK-4366] Enforce parallelism=1 For AllWindowedStream
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowed...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/2354
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/2354
Hi,
thanks for tackling this. I think the way to go about this is by subclassing `SingleOutputStreamOperator` and overriding `setParallelism` there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/2354
IMHO this looks good to merge now. \U0001f44d
@StephanEwen what do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/2354
Have to rerun tests on this one. The Scala API completeness check failed, so I had to adjust it.
Will merge pending successful tests...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Posted by wuchong <gi...@git.apache.org>.
Github user wuchong commented on the issue:
https://github.com/apache/flink/pull/2354
Thanks for advice. Do you have any ideas about the subclass's naming.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/2354
Merging this...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Posted by wuchong <gi...@git.apache.org>.
Github user wuchong commented on the issue:
https://github.com/apache/flink/pull/2354
Hi @aljoscha @StephanEwen , I add a `forceNonParallel` method to `SingleOutputStreamOperator`. I think subclassing SingleOutputStreamOperator is a little unnecessary.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/2354
Something like `NonParallelSingleOutputStreamOperator` but that's quite a mouthful. (`SingleOutputStreamOperator` is already too long for my taste ...). Or maybe `NonParallelStreamOperator`.
Maybe we could have it like you initially proposed but not named `allWindow` and in such a way that you cannot switch it back. For example: `SingleOutputStreamOperator.forceNonParallel()` that would set an internal flag that can never be unset from the outside again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/2354
I agree with Aljoscha, there may be more operators that need to constrain the parallelism. So a more generic solution would be good.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2354: [FLINK-4366] Enforce parallelism=1 For AllWindowedStream
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/2354
Yes, this is good, thanks!
+1 to merge this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---