You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by mlipkovich <gi...@git.apache.org> on 2017/09/07 14:08:21 UTC
[GitHub] flink pull request #4655: [FLINK-7567]: Removed keepPartitioning parameter f...
GitHub user mlipkovich opened a pull request:
https://github.com/apache/flink/pull/4655
[FLINK-7567]: Removed keepPartitioning parameter from iterate method
## What is the purpose of the change
Removed parameter keepPartitioning from DataStream#iterate method since it's ignored. Also slightly modified error message related to different parallelism levels of input and feedback streams
## Brief change log
- Removed parameter keepPartitioning from DataStream#iterate
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no) no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) yes
- The serializers: (yes / no / don't know) no
- The runtime per-record code paths (performance sensitive): (yes / no / don't know) no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know) no
## Documentation
- Does this pull request introduce a new feature? (yes / no) no
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) not applicable
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mlipkovich/flink FLINK-7567
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4655.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4655
----
commit 2525aef6f65d297142472ae6532e3bddb08df0fd
Author: Mikhail Lipkovich <mi...@epam.com>
Date: 2017-09-07T14:05:22Z
[FLINK-7567]: Removed keepPartitioning parameter from iterate method
----
---
[GitHub] flink issue #4655: [FLINK-7567]: Removed keepPartitioning parameter from ite...
Posted by mlipkovich <gi...@git.apache.org>.
Github user mlipkovich commented on the issue:
https://github.com/apache/flink/pull/4655
The local build works fine. If you mean `mvn verify` by compatibility plugin it also worked with no issues
---
[GitHub] flink issue #4655: [FLINK-7567]: Removed keepPartitioning parameter from ite...
Posted by mlipkovich <gi...@git.apache.org>.
Github user mlipkovich commented on the issue:
https://github.com/apache/flink/pull/4655
Thanks! Closing it
---
[GitHub] flink issue #4655: [FLINK-7567]: Removed keepPartitioning parameter from ite...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/4655
Thanks! This doesn't help us with our situation, though.
---
[GitHub] flink issue #4655: [FLINK-7567]: Removed keepPartitioning parameter from ite...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/4655
You can try and run the build locally to have a look at the file generated by the compatibility plugin. Then we can figure out why it's complaining.
---
[GitHub] flink issue #4655: [FLINK-7567]: Removed keepPartitioning parameter from ite...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/4655
A local build of `mvn clean verify` fails for me because the japicmp plugin is complaining. The relevant section of the diff output from japicmp is
```
***! MODIFIED CLASS: PUBLIC org.apache.flink.streaming.api.scala.DataStream (not serializable)
---! REMOVED METHOD: PUBLIC(-) boolean iterate$default$3()
+++ NEW METHOD: PUBLIC(+) org.apache.flink.streaming.api.scala.DataStream setMaxParallelism(int)
*** MODIFIED ANNOTATION: scala.reflect.ScalaSignature
```
i.e. it's complaining about `iterate$default$3()`. The problem seems to be that Scala will generate some obfuscated method name for the `iterate()` method and the `@PublicEvolving` annotation is not properly applied to that. I tried playing around with the japicmp config in the root pom file but to no avail. Maybe you can find something that works.
What I added is this:
``` <exclude>org.apache.flink.streaming.api.scala.DataStream#iterate$default$3()</exclude>
```
but it seems that doesn't work.
---
[GitHub] flink issue #4655: [FLINK-7567]: Removed keepPartitioning parameter from ite...
Posted by mlipkovich <gi...@git.apache.org>.
Github user mlipkovich commented on the issue:
https://github.com/apache/flink/pull/4655
Hi @aljoscha
It turned out that exclude filter works if we escape dollar signs. I commited these changes, but I believe it still should be fixed by japicmp
What do you think about merging this PR?
---
[GitHub] flink pull request #4655: [FLINK-7567]: Removed keepPartitioning parameter f...
Posted by mlipkovich <gi...@git.apache.org>.
Github user mlipkovich closed the pull request at:
https://github.com/apache/flink/pull/4655
---
[GitHub] flink issue #4655: [FLINK-7567]: Removed keepPartitioning parameter from ite...
Posted by mlipkovich <gi...@git.apache.org>.
Github user mlipkovich commented on the issue:
https://github.com/apache/flink/pull/4655
As I understand the build has failed because of changed API method. The method which was changed has annotation PublicEvolving so there should be a way to change it. As was mentioned by @aljoscha
https://issues.apache.org/jira/browse/FLINK-7567 there is no way to create a method with updated API and to deprecate the current one because of the default parameter
---
[GitHub] flink issue #4655: [FLINK-7567]: Removed keepPartitioning parameter from ite...
Posted by mlipkovich <gi...@git.apache.org>.
Github user mlipkovich commented on the issue:
https://github.com/apache/flink/pull/4655
Thansk, Aljoscha
Probably I've forgotten to run `clean`.
The method `iterate$default$3` is a method which automatically created by Scala for calculation of default value for parameter `keepPartitioning`. I tried different ways to exclude it but it didn't help. Anyway it should be somehow tracked by japicmp so I created the issue there
https://github.com/siom79/japicmp/issues/176
---
[GitHub] flink issue #4655: [FLINK-7567]: Removed keepPartitioning parameter from ite...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/4655
Perfect! Thanks for working on this. 👍
I merged, could you please close this PR?
---