You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by StephanEwen <gi...@git.apache.org> on 2014/07/08 12:25:43 UTC
[GitHub] incubator-flink pull request: [FLINK-1005] Add immutable object mo...
GitHub user StephanEwen opened a pull request:
https://github.com/apache/incubator-flink/pull/66
[FLINK-1005] Add immutable object mode utils and enable it for GroupReduce
This pull request adds the immutable object mode basics and implements it for the GroupReduce. This allows user code to keep references to values, without problems that the contents gets overwritten for mutable types.
I vote to make this the default mode in future versions.
Code like this used to give unexpected results in the past, because of heavy object reuse in the runtime. With *immutable object mode*, it now gives expected results.
```
List<Tuple2<StringValue, IntValue>> all = new ArrayList<Tuple2<StringValue,IntValue>>();
while (values.hasNext()) {
all.add(values.next());
}
Tuple2<StringValue, IntValue> result = all.get(0);
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink mutable_immutable
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/66.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #66
----
commit 0f5c049cc3f91530ca36816d9742c2a04234989f
Author: Stephan Ewen <se...@apache.org>
Date: 2014-07-07T17:39:24Z
[FLINK-1005] Extend TypeSerializer interface to handle non-mutable object deserialization more efficiently.
commit 618e0b3d7c72b67a555a1e8db6925a7d5d0b4c92
Author: Stephan Ewen <se...@apache.org>
Date: 2014-07-08T09:32:09Z
[FLINK-1005] Add non-object reusing variants of key-grouped iterator.
Clean minor javadoc errors.
commit 9703593a4b35b148489e840875b46b45e26bf966
Author: Stephan Ewen <se...@apache.org>
Date: 2014-07-08T10:19:43Z
[FLINK-1005] Make GroupReduce configurable to use either mutable or immutable object mode
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1005] Add immutable object mo...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/incubator-flink/pull/66#issuecomment-48526517
Very nice. This is very helpful for new users and reduces frustration when trying out the system.
Once its merged, I'll run some benchmarks on the cluster to better understand the performance implications.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1005] Add immutable object mo...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen closed the pull request at:
https://github.com/apache/incubator-flink/pull/66
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1005] Add immutable object mo...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen closed the pull request at:
https://github.com/apache/incubator-flink/pull/66
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1005] Add immutable object mo...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the pull request:
https://github.com/apache/incubator-flink/pull/66#issuecomment-48320218
+1 to have *immutable object mode* as the default and then document how you can and why you would switch to *mutable mode*.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1005] Add immutable object mo...
Posted by StephanEwen <gi...@git.apache.org>.
GitHub user StephanEwen reopened a pull request:
https://github.com/apache/incubator-flink/pull/66
[FLINK-1005] Add immutable object mode utils and enable it for GroupReduce
This pull request adds the immutable object mode basics and implements it for the GroupReduce. This allows user code to keep references to values, without problems that the contents gets overwritten for mutable types.
I vote to make this the default mode in future versions.
Code like this used to give unexpected results in the past, because of heavy object reuse in the runtime. With *immutable object mode*, it now gives expected results.
```
List<Tuple2<StringValue, IntValue>> all = new ArrayList<Tuple2<StringValue,IntValue>>();
while (values.hasNext()) {
all.add(values.next());
}
Tuple2<StringValue, IntValue> result = all.get(0);
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink mutable_immutable
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/66.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #66
----
commit 0f5c049cc3f91530ca36816d9742c2a04234989f
Author: Stephan Ewen <se...@apache.org>
Date: 2014-07-07T17:39:24Z
[FLINK-1005] Extend TypeSerializer interface to handle non-mutable object deserialization more efficiently.
commit 618e0b3d7c72b67a555a1e8db6925a7d5d0b4c92
Author: Stephan Ewen <se...@apache.org>
Date: 2014-07-08T09:32:09Z
[FLINK-1005] Add non-object reusing variants of key-grouped iterator.
Clean minor javadoc errors.
commit 9703593a4b35b148489e840875b46b45e26bf966
Author: Stephan Ewen <se...@apache.org>
Date: 2014-07-08T10:19:43Z
[FLINK-1005] Make GroupReduce configurable to use either mutable or immutable object mode
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1005] Add immutable object mo...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/incubator-flink/pull/66#discussion_r14732236
--- Diff: stratosphere-runtime/src/main/java/eu/stratosphere/pact/runtime/task/util/TaskConfig.java ---
@@ -68,6 +68,8 @@
private static final String DRIVER_COMPARATOR_PARAMETERS_PREFIX = "driver.comp.params.";
private static final String DRIVER_PAIR_COMPARATOR_FACTORY = "driver.paircomp";
+
+ private static final String DRIVER_MUTABLE_OBJECT_MODE = "diver.mutableobjects";
--- End diff --
There is a typo in `diver`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---