You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2015/02/13 01:01:34 UTC
[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/392
[FLINK-1521] Chained operators respect reuse
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/incubator-flink flink-1521
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/392.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #392
----
commit b3f8ec9a107a2baf743468c9efdf28da9f49cf79
Author: zentol <s....@web.de>
Date: 2015-02-12T19:36:00Z
[FLINK-1521] Chained operators respect reuse
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse
Posted by gyfora <gi...@git.apache.org>.
Github user gyfora commented on the pull request:
https://github.com/apache/flink/pull/392#issuecomment-74367347
The problem here I think that an error caused by the reusing mapper could be very hard to detect. So some users might have it but they don't realise.
For streaming we copy the input at every chained function call just to be on the safe side, but we'll have to reintroduce the object reuse mode.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/392#issuecomment-74237302
I agree with Stephan to not copy records for chained mappers.
However, we should clearly document the behavior of the different modes or at least the behavior of the default mode.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/392#issuecomment-74565272
At this point, we also have to worry about efficiency and performance.
This is also something that the user can easily mitigate in the UDF, at a fraction of the cost in most cases.
I vote postpone the fix until we get some tangible user feedback concerning this issue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/392#issuecomment-74650731
+1
I opened FLINK-1565 for documenting object reuse behavior.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/392#issuecomment-74234126
While this change makes it technically more correct, I think it will have a massive negative impact on performance. Nobody can be expected to turn on the object reuse switch without being a seriously advanced user. But everybody will experience the performance drop.
Since no user ever ran into this problem, I think we should change it at this point.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the pull request:
https://github.com/apache/flink/pull/392#issuecomment-75029399
Alright, I`m closing this PR then.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1521] Chained operators respect reuse
Posted by zentol <gi...@git.apache.org>.
Github user zentol closed the pull request at:
https://github.com/apache/flink/pull/392
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---