You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by jinossy <gi...@git.apache.org> on 2015/11/19 10:38:59 UTC
[GitHub] tajo pull request: TAJO-1983: Improve memory usage of ExternalSort...
GitHub user jinossy opened a pull request:
https://github.com/apache/tajo/pull/869
TAJO-1983: Improve memory usage of ExternalSortExec.
I've add UnSafeTupleList. it uses off-heap memory.
A last chunk are not stored in local file and the chunks are merged when the next method is called.
In addition, I’ve change the insignificant log level to debug.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jinossy/tajo TAJO-1983
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/869.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #869
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1983: Improve memory usage of ExternalSort...
Posted by hyunsik <gi...@git.apache.org>.
Github user hyunsik commented on the pull request:
https://github.com/apache/tajo/pull/869#issuecomment-159773488
The patch looks great to me. Here is my +1. I leaved some trivial comments.You can commit the patch if you reflect my comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1983: Improve memory usage of ExternalSort...
Posted by jinossy <gi...@git.apache.org>.
Github user jinossy commented on the pull request:
https://github.com/apache/tajo/pull/869#issuecomment-159139295
This PR is ready to review.
Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1983: Improve memory usage of ExternalSort...
Posted by hyunsik <gi...@git.apache.org>.
Github user hyunsik commented on a diff in the pull request:
https://github.com/apache/tajo/pull/869#discussion_r45936295
--- Diff: tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/ExternalSortExec.java ---
@@ -103,7 +117,7 @@
/** the final result */
--- End diff --
The member variable ``memoryResident`` seems to be not necessary anymore.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1983: Improve memory usage of ExternalSort...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/tajo/pull/869
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1983: Improve memory usage of ExternalSort...
Posted by hyunsik <gi...@git.apache.org>.
Github user hyunsik commented on a diff in the pull request:
https://github.com/apache/tajo/pull/869#discussion_r45935202
--- Diff: tajo-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java ---
@@ -403,12 +400,20 @@ public void decrementRemainFiles(FileRegion filePart, long fileStartTime) {
minTime = fileSendTime;
}
+ if (fileSendTime > 20 * 1000) {
+ LOG.warn("PullServer send too long time: filePos:" + filePart.position()
--- End diff --
I know you just moved this code. But, the original logging message needs to be corrected. Could you update the log message?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1983: Improve memory usage of ExternalSort...
Posted by jinossy <gi...@git.apache.org>.
Github user jinossy commented on the pull request:
https://github.com/apache/tajo/pull/869#issuecomment-159787856
I've update the patch that reflects your comments and I will commit soon
Thanks for your review!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---