You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by tillrohrmann <gi...@git.apache.org> on 2015/11/04 02:20:28 UTC
[GitHub] flink pull request: [FLINK-2964] [runtime] Fix broken spilling of ...
GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/1324
[FLINK-2964] [runtime] Fix broken spilling of MutableHashTable
The `HashPartition` did not count properly the number of occupied memory segments, because it excluded the memory segments of the `BuildSideBuffer`. That caused the `MutableHashTable` to fail when trying to spill a partition which did not have any overflow segments. This PR fixes the problem by also counting the memory segments of the `BuildSideBuffer`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink fixMutableHashTable
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1324.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1324
----
commit 9db81acd4012cae2a2fc845bc762ab7082463ba2
Author: Till Rohrmann <tr...@apache.org>
Date: 2015-11-04T01:17:12Z
[FLINK-2964] [runtime] Fix broken spilling of MutableHashTable
The HashPartition did not count properly the number of occupied memory segments, because it excluded the memory segments of the BuildSideBuffer. That caused the MutableHashTable to fail when trying to spill a partition which did not have any overflow segments. This PR fixes the problem by also counting the memory segments of the BuildSideBuffer.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2964] [runtime] Fix broken spilling of ...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1324#issuecomment-153671502
Merging..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2964] [runtime] Fix broken spilling of ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/1324
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2964] [runtime] Fix broken spilling of ...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/1324#issuecomment-153669167
Important fix!
+1 to merge and include in the next RC
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2964] [runtime] Fix broken spilling of ...
Posted by fpompermaier <gi...@git.apache.org>.
Github user fpompermaier commented on the pull request:
https://github.com/apache/flink/pull/1324#issuecomment-153606566
Do yiu think that this could fix also #2763?
On 4 Nov 2015 02:20, "Till Rohrmann" <no...@github.com> wrote:
> The HashPartition did not count properly the number of occupied memory
> segments, because it excluded the memory segments of the BuildSideBuffer.
> That caused the MutableHashTable to fail when trying to spill a partition
> which did not have any overflow segments. This PR fixes the problem by also
> counting the memory segments of the BuildSideBuffer.
> ------------------------------
> You can view, comment on, or merge this pull request online at:
>
> https://github.com/apache/flink/pull/1324
> Commit Summary
>
> - [FLINK-2964] [runtime] Fix broken spilling of MutableHashTable
>
> File Changes
>
> - *M*
> flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/HashPartition.java
> <https://github.com/apache/flink/pull/1324/files#diff-0> (8)
> - *M*
> flink-runtime/src/test/java/org/apache/flink/runtime/operators/hash/HashTableTest.java
> <https://github.com/apache/flink/pull/1324/files#diff-1> (124)
>
> Patch Links:
>
> - https://github.com/apache/flink/pull/1324.patch
> - https://github.com/apache/flink/pull/1324.diff
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/flink/pull/1324>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2964] [runtime] Fix broken spilling of ...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1324#issuecomment-153652855
+1 very good fix. I would like to merge it for the next RC.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---