You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by hyunsik <gi...@git.apache.org> on 2014/08/01 05:57:20 UTC
[GitHub] tajo pull request: TJAO-987: Hash shuffle should be balanced accor...
GitHub user hyunsik opened a pull request:
https://github.com/apache/tajo/pull/101
TJAO-987: Hash shuffle should be balanced according to intermediate volumes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hyunsik/tajo TAJO-987
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/101.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #101
----
commit 06b3153418692c8dccf692e6486b9396074ebb5a
Author: Hyunsik Choi <hy...@apache.org>
Date: 2014-08-01T00:24:20Z
Add scheduleFetchesByEvenDistributedVolumes.
commit bdd14ddff5b71924dd56098dbb76c5babf64111a
Author: Hyunsik Choi <hy...@apache.org>
Date: 2014-08-01T01:26:36Z
Improve the shuffle handlers.
* Add unit tests to verify evenly distributed shuffles
* Improve it to result in more fine grained distribution
commit ba65417fcc07c701d0290b523ed48a8af59bff66
Author: Hyunsik Choi <hy...@apache.org>
Date: 2014-08-01T03:56:05Z
Fixed a bug to loss some partitions.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-987: Hash shuffle should be balanced accor...
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/101#issuecomment-51012246
+1
this patch looks good to me.
As I commented, please remove some unused imports before commit.
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-987: Hash shuffle should be balanced accor...
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/101#issuecomment-51010170
Sorry for late review.
I'll review today.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-987: Hash shuffle should be balanced accor...
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on a diff in the pull request:
https://github.com/apache/tajo/pull/101#discussion_r15738646
--- Diff: tajo-core/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java ---
@@ -18,7 +18,9 @@
package org.apache.tajo.master.querymaster;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
--- End diff --
This is an unused imports.
Please remove it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-987: Hash shuffle should be balanced accor...
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on a diff in the pull request:
https://github.com/apache/tajo/pull/101#discussion_r15738637
--- Diff: tajo-core/src/test/java/org/apache/tajo/master/TestRepartitioner.java ---
@@ -18,15 +18,21 @@
package org.apache.tajo.master;
+import com.google.common.collect.Maps;
import org.apache.tajo.ExecutionBlockId;
import org.apache.tajo.QueryId;
import org.apache.tajo.TestTajoIds;
import org.apache.tajo.ipc.TajoWorkerProtocol;
+import org.apache.tajo.master.querymaster.QueryMaster;
import org.apache.tajo.master.querymaster.QueryUnit;
+import org.apache.tajo.master.querymaster.Repartitioner;
+import org.apache.tajo.master.querymaster.SubQuery;
+import org.apache.tajo.util.Pair;
import org.apache.tajo.util.TUtil;
import org.apache.tajo.worker.FetchImpl;
import org.jboss.netty.handler.codec.http.QueryStringDecoder;
import org.junit.Test;
+import org.mockito.Mockito;
--- End diff --
There are some unused imports.
Please remove them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-987: Hash shuffle should be balanced accor...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/tajo/pull/101
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---