You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by jihoonson <gi...@git.apache.org> on 2015/07/27 05:59:17 UTC
[GitHub] tajo pull request: TAJO-1552: NPE occurs when GreedyHeuristicJoinO...
GitHub user jihoonson opened a pull request:
https://github.com/apache/tajo/pull/655
TAJO-1552: NPE occurs when GreedyHeuristicJoinOrderAlgorithm.getCost() returns infinity
This patch is based on #547.
Changes are:
* Rebasing the work of #547
* Removed an unnecessary test
* Added a ```FAKEFILE``` store type for test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jihoonson/tajo-2 TAJO-1552
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/655.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #655
----
commit 14dffcc60794c2c9c560a10c8d206afa553de77c
Author: Hyoungjun Kim <ba...@gmail.com>
Date: 2015-04-20T17:51:50Z
TAJO-1552: NPE occurs when GreedyHeuristicJoinOrderAlgorithm.getCost() returns infinity.
commit ed2943de1af55683bd942e8f42dc20f85208aad8
Author: Hyoungjun Kim <ba...@gmail.com>
Date: 2015-04-20T17:54:43Z
TAJO-1552: NPE occurs when GreedyHeuristicJoinOrderAlgorithm.getCost() returns infinity.
commit 7d66bdee67bd0e1245f8d9426c56e8879b66c540
Author: Hyoungjun Kim <ba...@gmail.com>
Date: 2015-04-20T17:55:33Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo
commit d674b6567f35ed3a97682197d31d4b883710f520
Author: Hyoungjun Kim <ba...@gmail.com>
Date: 2015-04-20T18:26:52Z
TAJO-1552: NPE occurs when GreedyHeuristicJoinOrderAlgorithm.getCost() returns infinity.
Rename TestGreedyHeuristicJoinOrderAlgorithm to TestJoinOrderAlgorithm.
commit ac5dcc024b759a6f2c85fd8f9bef8334f340addd
Author: Hyoungjun Kim <ba...@gmail.com>
Date: 2015-05-11T10:49:31Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1552
commit 0b05727faecf82b8e8a0e77de848ef3edffa635c
Author: Hyoungjun Kim <ba...@gmail.com>
Date: 2015-05-11T14:37:34Z
TAJO-1552: NPE occurs when GreedyHeuristicJoinOrderAlgorithm.getCost() returns infinity.
commit 846747bfb6aac6c6f1dc18ed3b9ea8b993a15901
Author: Jihoon Son <ji...@apache.org>
Date: 2015-07-24T09:47:03Z
TAJO-1552
commit 70d4987ff32136e85b4fe4fd4cd5c0cb65694162
Author: Jihoon Son <ji...@apache.org>
Date: 2015-07-27T02:42:41Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1552
commit 297f5d8e3f78f33136a1b482c93b0461a552a2a1
Author: Jihoon Son <ji...@apache.org>
Date: 2015-07-27T03:55:09Z
TAJO-1552
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1552: NPE occurs when GreedyHeuristicJoinO...
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/655#issuecomment-125897601
Thanks for your review!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1552: NPE occurs when GreedyHeuristicJoinO...
Posted by hyunsik <gi...@git.apache.org>.
Github user hyunsik commented on the pull request:
https://github.com/apache/tajo/pull/655#issuecomment-125881403
LGTM +1
You can reflect my trivial comments before committing it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1552: NPE occurs when GreedyHeuristicJoinO...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/tajo/pull/655
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1552: NPE occurs when GreedyHeuristicJoinO...
Posted by hyunsik <gi...@git.apache.org>.
Github user hyunsik commented on a diff in the pull request:
https://github.com/apache/tajo/pull/655#discussion_r35738078
--- Diff: tajo-core/src/test/java/org/apache/tajo/engine/planner/TestJoinOrderAlgorithm.java ---
@@ -0,0 +1,184 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tajo.engine.planner;
+
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.tajo.LocalTajoTestingUtility;
+import org.apache.tajo.TajoConstants;
+import org.apache.tajo.TajoTestingCluster;
+import org.apache.tajo.algebra.Expr;
+import org.apache.tajo.catalog.*;
+import org.apache.tajo.catalog.proto.CatalogProtos.StoreType;
+import org.apache.tajo.catalog.statistics.TableStats;
+import org.apache.tajo.common.TajoDataTypes.Type;
+import org.apache.tajo.engine.function.FunctionLoader;
+import org.apache.tajo.engine.parser.SQLAnalyzer;
+import org.apache.tajo.engine.query.QueryContext;
+import org.apache.tajo.plan.LogicalOptimizer;
+import org.apache.tajo.plan.LogicalPlan;
+import org.apache.tajo.plan.LogicalPlanner;
+import org.apache.tajo.plan.logical.*;
+import org.apache.tajo.plan.util.PlannerUtil;
+import org.apache.tajo.storage.TablespaceManager;
+import org.apache.tajo.util.CommonTestingUtil;
+import org.apache.tajo.util.KeyValueSet;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static org.apache.tajo.TajoConstants.DEFAULT_DATABASE_NAME;
+import static org.apache.tajo.TajoConstants.DEFAULT_TABLESPACE_NAME;
+import static org.junit.Assert.*;
+
+public class TestJoinOrderAlgorithm {
+
+ private static TajoTestingCluster util;
+ private static CatalogService catalog;
+ private static SQLAnalyzer sqlAnalyzer;
+ private static LogicalPlanner planner;
+ private static LogicalOptimizer optimizer;
+ private static QueryContext defaultContext;
+
+ @BeforeClass
+ public static void setUp() throws Exception {
+ util = new TajoTestingCluster();
+ util.startCatalogCluster();
+ catalog = util.getMiniCatalogCluster().getCatalog();
+ catalog.createTablespace(DEFAULT_TABLESPACE_NAME, "hdfs://localhost:1234/warehouse");
+ catalog.createDatabase(DEFAULT_DATABASE_NAME, DEFAULT_TABLESPACE_NAME);
+ for (FunctionDesc funcDesc : FunctionLoader.findLegacyFunctions()) {
+ catalog.createFunction(funcDesc);
+ }
+
+ Schema schema = new Schema();
+ schema.addColumn("name", Type.TEXT);
+ schema.addColumn("empid", Type.INT4);
+ schema.addColumn("deptname", Type.TEXT);
+
+ Schema schema2 = new Schema();
+ schema2.addColumn("deptname", Type.TEXT);
+ schema2.addColumn("manager", Type.TEXT);
+
+ Schema schema3 = new Schema();
+ schema3.addColumn("deptname", Type.TEXT);
+ schema3.addColumn("score", Type.INT4);
+ schema3.addColumn("phone", Type.INT4);
+
+ TableMeta meta = CatalogUtil.newTableMeta("TEXT");
+ TableDesc people = new TableDesc(
+ CatalogUtil.buildFQName(TajoConstants.DEFAULT_DATABASE_NAME, "employee"), schema, meta,
+ CommonTestingUtil.getTestDir().toUri());
+ catalog.createTable(people);
+
+ TableDesc student =
+ new TableDesc(
+ CatalogUtil.buildFQName(DEFAULT_DATABASE_NAME, "dept"), schema2, "TEXT", new KeyValueSet(),
+ CommonTestingUtil.getTestDir().toUri());
+ catalog.createTable(student);
+
+ TableDesc score =
+ new TableDesc(
+ CatalogUtil.buildFQName(DEFAULT_DATABASE_NAME, "score"), schema3, "TEXT", new KeyValueSet(),
+ CommonTestingUtil.getTestDir().toUri());
+ catalog.createTable(score);
+
+ ///////////////////////////////////////////////////////////////////////////
+ // creating table for overflow in JoinOrderOptimizer.
+ Schema schema4 = new Schema();
+ schema4.addColumn("deptname", Type.TEXT);
+ schema4.addColumn("manager", Type.TEXT);
+ // Set store type as FAKEFILE to prevent auto update of physical information in LogicalPlanner.updatePhysicalInfo()
+ TableMeta largeTableMeta = CatalogUtil.newTableMeta("FAKEFILE");
+ TableDesc largeDept;
+ TableStats largeTableStats;
+ FileSystem fs = FileSystem.getLocal(util.getConfiguration());
+ for (int i = 0; i < 6; i++) {
+ Path tablePath = new Path(CommonTestingUtil.getTestDir(), "" + (i+1));
+ fs.create(tablePath);
+ largeDept =
+ new TableDesc(
+ CatalogUtil.buildFQName(DEFAULT_DATABASE_NAME, "large_dept"+(i+1)), schema4, largeTableMeta,
+ tablePath.toUri());
+ largeTableStats = new TableStats();
+ largeTableStats.setNumBytes(1024L * 1024L * 1024L * 1024L * 1024L * (i+1)); //1 PB * i
--- End diff --
You may use StorageUnit.PB?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---