You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by maryannxue <gi...@git.apache.org> on 2014/04/22 04:51:47 UTC
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
GitHub user maryannxue opened a pull request:
https://github.com/apache/incubator-phoenix/pull/31
PHOENIX-136 Support derived tables
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/maryannxue/incubator-phoenix master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-phoenix/pull/31.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #31
----
commit 05b584181fe36102feca765886fcda75b770b2b2
Author: maryannxue <ma...@apache.org>
Date: 2014-02-12T21:41:53Z
Fix PHOENIX-44 Cannot create index on tables with case-sensitive schema names
commit 02af57c96476c9d205d07ba96974c49188a282fa
Author: maryannxue <ma...@apache.org>
Date: 2014-02-21T02:54:17Z
Merge remote-tracking branch 'asf/master'
commit bd20346da5336d3d240493fdbfbcd5cc03280810
Author: maryannxue <ma...@apache.org>
Date: 2014-02-21T03:08:23Z
PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries
commit 9a2ced8936d6cedbc44a4ed2de2c6f72f4f4307f
Author: maryannxue <ma...@apache.org>
Date: 2014-02-21T04:59:29Z
PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries
commit 20a97904e48829f8b412bcfb0ac2084259133126
Author: maryannxue <ma...@apache.org>
Date: 2014-02-21T05:53:21Z
PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries
commit 7568516fabfb416f5dcbfa89468b839963802a87
Author: maryannxue <ma...@apache.org>
Date: 2014-02-21T22:14:14Z
Revert "PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries"
This reverts commit 20a97904e48829f8b412bcfb0ac2084259133126.
commit 3fb95819959d22deb2c03e895d976e7d933007f2
Author: maryannxue <ma...@apache.org>
Date: 2014-02-21T22:14:56Z
Revert "PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries"
This reverts commit 9a2ced8936d6cedbc44a4ed2de2c6f72f4f4307f.
commit ca0b55d048a6c547a5d6b081dd84dea01011e5c1
Author: maryannxue <ma...@apache.org>
Date: 2014-02-21T22:16:13Z
Revert "PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries"
This reverts commit bd20346da5336d3d240493fdbfbcd5cc03280810.
commit c4df65791a163070362662aad7ce5916faa3c3d7
Author: maryannxue <ma...@apache.org>
Date: 2014-02-21T22:18:13Z
Merge remote-tracking branch 'asf/master'
commit 9b5df132480b6bd25c0c33dd818278a5fc1b3f55
Author: maryannxue <ma...@apache.org>
Date: 2014-02-21T22:22:38Z
PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries
commit 3e677489c0cb811c1bb3d266581330fe11fdd674
Author: maryannxue <ma...@apache.org>
Date: 2014-02-22T00:17:15Z
PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries
commit 776c1a11b286ebd358f9111f3ff1c99c3f7c7883
Author: maryannxue <ma...@apache.org>
Date: 2014-02-27T18:41:58Z
Merge remote-tracking branch 'asf/master'
commit 6d26e78096091624502d4382de23f655626bca58
Author: maryannxue <ma...@apache.org>
Date: 2014-02-28T19:36:38Z
Merge remote-tracking branch 'asf/master'
commit df561c8d8ab59651d7e797dc0da501cceb4c6030
Author: maryannxue <ma...@apache.org>
Date: 2014-02-28T19:48:06Z
PHOENIX-85 Refine error handling in HashJoinPlan execution and add warnings
commit 53631fa509e2a660241c717a4eb0c131b251ff5d
Author: maryannxue <ma...@apache.org>
Date: 2014-02-28T23:42:14Z
PHOENIX-85 Refine error handling in HashJoinPlan execution and add warnings
commit 4fb71128f6ac961acbba7db9c3041c53bd18a64a
Author: maryannxue <ma...@apache.org>
Date: 2014-03-01T01:53:26Z
Merge remote-tracking branch 'asf/master'
commit 6a800b53abc6b6dac24daff1eae74100d6c73b97
Author: maryannxue <ma...@apache.org>
Date: 2014-03-01T02:10:20Z
Merge remote-tracking branch 'asf/master'
commit af32148215ec4cb443ba9d79c705c94196f1ff03
Author: maryannxue <ma...@apache.org>
Date: 2014-03-03T01:26:48Z
Merge remote-tracking branch 'asf/master'
commit 012018d2f58cfb35f7574301c68314ead2f31b9d
Author: maryannxue <ma...@apache.org>
Date: 2014-03-03T01:30:49Z
PHOENIX-89 Statements are not closed using PhoenixConnection.executeStatements()
commit c4bd34ff11f75d00bb74008a03247889b3b0160a
Author: maryannxue <ma...@apache.org>
Date: 2014-03-31T01:52:33Z
Merge remote-tracking branch 'asf/master'
commit 8fce312e63138f23c1befcf3a7dbbb573275eaf1
Author: maryannxue <ma...@apache.org>
Date: 2014-03-31T01:59:00Z
PHOENIX-71 Support sub-joins
commit b1d9340e36f750c66b993eb5e8571371f36ca9f9
Author: maryannxue <ma...@apache.org>
Date: 2014-04-01T00:19:57Z
PHOENIX-71 Support sub-joins
commit a3c8f7572a5aa3704ec817f5c42d691df2b631f6
Author: maryannxue <ma...@apache.org>
Date: 2014-04-01T00:43:04Z
PHOENIX-905 Wrong calculation of maxSetBit in ValueBitSet#or(ImmutableBytesWritable)
commit 2a99a6d5062a37eb654796550347786195b91b89
Author: maryannxue <ma...@apache.org>
Date: 2014-04-01T00:50:22Z
PHOENIX-905 Wrong calculation of maxSetBit in ValueBitSet#or(ImmutableBytesWritable)
commit 860a70d2fe08d16bba7a412810feae957089b39b
Author: maryannxue <ma...@apache.org>
Date: 2014-04-01T17:44:32Z
PHOENIX-905 Wrong calculation of maxSetBit in ValueBitSet#or(ImmutableBytesWritable)
commit 037779bd1de9409e741bd97a03c55cec072106c8
Author: maryannxue <ma...@apache.org>
Date: 2014-04-10T18:19:06Z
Merge remote-tracking branch 'asf/master'
commit 71569be703d185e5fc6102423ab15aaa3361cb77
Author: maryannxue <ma...@apache.org>
Date: 2014-04-10T18:37:14Z
PHOENIX-927 Support derived tables in joins
commit 47d9fd827e4f47795ce5f4bfa0c5116ebd6b1e16
Author: maryannxue <ma...@apache.org>
Date: 2014-04-13T20:42:27Z
PHOENIX-927 Support derived tables in joins
commit a28a9f454c31e50a0b3cfe974ccf894b8dfd19d2
Author: maryannxue <ma...@apache.org>
Date: 2014-04-22T02:35:29Z
Merge remote-tracking branch 'asf/master'
commit ffcc2715f69e3b2c3a63b368656fc02b60561935
Author: maryannxue <ma...@apache.org>
Date: 2014-04-22T02:47:47Z
PHOENIX-136 Support derived tables
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11857151
--- Diff: phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java ---
@@ -0,0 +1,632 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.phoenix.end2end;
+
+import static org.apache.phoenix.util.TestUtil.A_VALUE;
+import static org.apache.phoenix.util.TestUtil.B_VALUE;
+import static org.apache.phoenix.util.TestUtil.C_VALUE;
+import static org.apache.phoenix.util.TestUtil.E_VALUE;
+import static org.apache.phoenix.util.TestUtil.ROW1;
+import static org.apache.phoenix.util.TestUtil.ROW2;
+import static org.apache.phoenix.util.TestUtil.ROW3;
+import static org.apache.phoenix.util.TestUtil.ROW4;
+import static org.apache.phoenix.util.TestUtil.ROW5;
+import static org.apache.phoenix.util.TestUtil.ROW6;
+import static org.apache.phoenix.util.TestUtil.ROW7;
+import static org.apache.phoenix.util.TestUtil.ROW8;
+import static org.apache.phoenix.util.TestUtil.ROW9;
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLFeatureNotSupportedException;
+import java.util.Properties;
+
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.junit.Test;
+
+public class DerivedTableIT extends BaseClientManagedTimeIT {
--- End diff --
Ok, I'll make it parameterized and keep it separate as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue commented on the pull request:
https://github.com/apache/incubator-phoenix/pull/31#issuecomment-41050145
Sure! @JamesRTaylor.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11864465
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/compile/QueryCompiler.java ---
@@ -290,6 +292,9 @@ protected BasicQueryPlan compileSingleQuery(StatementContext context, SelectStat
ColumnResolver resolver = context.getResolver();
TableRef tableRef = context.getCurrentTable();
PTable table = tableRef.getTable();
+ if (table.getType() == PTableType.SUBQUERY)
+ throw new SQLFeatureNotSupportedException("Complex nested queries not supported.");
--- End diff --
I will file a JIRA. As to examples, actually all those counter cases in DerivedTableIT throw exception from here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11862953
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/compile/SubselectRewriter.java ---
@@ -0,0 +1,242 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.phoenix.compile;
+
+import java.sql.SQLException;
+import java.sql.SQLFeatureNotSupportedException;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.parse.AliasedNode;
+import org.apache.phoenix.parse.ColumnParseNode;
+import org.apache.phoenix.parse.DerivedTableNode;
+import org.apache.phoenix.parse.HintNode;
+import org.apache.phoenix.parse.LimitNode;
+import org.apache.phoenix.parse.OrderByNode;
+import org.apache.phoenix.parse.ParseNode;
+import org.apache.phoenix.parse.ParseNodeRewriter;
+import org.apache.phoenix.parse.SelectStatement;
+import org.apache.phoenix.parse.TableNode;
+import org.apache.phoenix.parse.TableWildcardParseNode;
+import org.apache.phoenix.parse.WildcardParseNode;
+import org.apache.phoenix.util.SchemaUtil;
+
+import com.google.common.collect.Lists;
+
+public class SubselectRewriter extends ParseNodeRewriter {
+ private final String tableAlias;
+ private final Map<String, ParseNode> aliasMap;
+
+ public static SelectStatement applyPostFilters(SelectStatement statement, List<ParseNode> postFilters, String subqueryAlias) throws SQLException {
+ if (postFilters.isEmpty())
+ return statement;
+
+ // TODO handle this from caller
+ if (statement.getLimit() != null || (statement.isAggregate() && statement.getGroupBy().isEmpty()))
--- End diff --
Do you think it's feasible to do another set of changes for the compiler to handle the cases it can for derived tables and joins?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11862513
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/compile/SubselectRewriter.java ---
@@ -0,0 +1,242 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.phoenix.compile;
+
+import java.sql.SQLException;
+import java.sql.SQLFeatureNotSupportedException;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.parse.AliasedNode;
+import org.apache.phoenix.parse.ColumnParseNode;
+import org.apache.phoenix.parse.DerivedTableNode;
+import org.apache.phoenix.parse.HintNode;
+import org.apache.phoenix.parse.LimitNode;
+import org.apache.phoenix.parse.OrderByNode;
+import org.apache.phoenix.parse.ParseNode;
+import org.apache.phoenix.parse.ParseNodeRewriter;
+import org.apache.phoenix.parse.SelectStatement;
+import org.apache.phoenix.parse.TableNode;
+import org.apache.phoenix.parse.TableWildcardParseNode;
+import org.apache.phoenix.parse.WildcardParseNode;
+import org.apache.phoenix.util.SchemaUtil;
+
+import com.google.common.collect.Lists;
+
+public class SubselectRewriter extends ParseNodeRewriter {
+ private final String tableAlias;
+ private final Map<String, ParseNode> aliasMap;
+
+ public static SelectStatement applyPostFilters(SelectStatement statement, List<ParseNode> postFilters, String subqueryAlias) throws SQLException {
+ if (postFilters.isEmpty())
+ return statement;
+
+ // TODO handle this from caller
+ if (statement.getLimit() != null || (statement.isAggregate() && statement.getGroupBy().isEmpty()))
--- End diff --
Right, I saw that test case. So under what circumstances would statement.getLimit() != null be true? I think it'd be good to capture the cases we're currently not handling in code comments and in JIRAs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11866024
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/compile/SubselectRewriter.java ---
@@ -0,0 +1,242 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.phoenix.compile;
+
+import java.sql.SQLException;
+import java.sql.SQLFeatureNotSupportedException;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.parse.AliasedNode;
+import org.apache.phoenix.parse.ColumnParseNode;
+import org.apache.phoenix.parse.DerivedTableNode;
+import org.apache.phoenix.parse.HintNode;
+import org.apache.phoenix.parse.LimitNode;
+import org.apache.phoenix.parse.OrderByNode;
+import org.apache.phoenix.parse.ParseNode;
+import org.apache.phoenix.parse.ParseNodeRewriter;
+import org.apache.phoenix.parse.SelectStatement;
+import org.apache.phoenix.parse.TableNode;
+import org.apache.phoenix.parse.TableWildcardParseNode;
+import org.apache.phoenix.parse.WildcardParseNode;
+import org.apache.phoenix.util.SchemaUtil;
+
+import com.google.common.collect.Lists;
+
+public class SubselectRewriter extends ParseNodeRewriter {
+ private final String tableAlias;
+ private final Map<String, ParseNode> aliasMap;
+
+ public static SelectStatement applyPostFilters(SelectStatement statement, List<ParseNode> postFilters, String subqueryAlias) throws SQLException {
+ if (postFilters.isEmpty())
+ return statement;
+
+ // TODO handle this from caller
+ if (statement.getLimit() != null || (statement.isAggregate() && statement.getGroupBy().isEmpty()))
--- End diff --
This applyPostFilter() method is for subqueries in joins. We can have some conditions in ON clause or WHERE clause to be pushed down into individual join tables, and if the join table is a subquery, we call this method to merge this post-filter into the subquery, as a WHERE condition or a HAVING condition so that in some cases these post-filters can be optimized as pre-filters.
Those cases that would throw exceptions from here are very rare, basically like:
1) select ... from A join (select id, b from T limit 10) as B on A.id = B.id where B.b = 'b'
2) select ... from A join (select count(*) c from T) as B on A.a = B.c where B.c > 10
However, this post-filter can be easily handled in joins. I will make another check-in later.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on the pull request:
https://github.com/apache/incubator-phoenix/pull/31#issuecomment-41003342
+1. This is fantastic, @maryannxue! Let's file separate JIRAs for each type of query that's not yet supported. The list is getting smaller!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11857047
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/compile/QueryCompiler.java ---
@@ -290,6 +292,9 @@ protected BasicQueryPlan compileSingleQuery(StatementContext context, SelectStat
ColumnResolver resolver = context.getResolver();
TableRef tableRef = context.getCurrentTable();
PTable table = tableRef.getTable();
+ if (table.getType() == PTableType.SUBQUERY)
+ throw new SQLFeatureNotSupportedException("Complex nested queries not supported.");
--- End diff --
No, this is just subqueries (derived tables) in the from clause that SubselectRewriter is unable to flatten. We don't have parser support for correlated subqueries so far.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11838107
--- Diff: phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java ---
@@ -0,0 +1,632 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.phoenix.end2end;
+
+import static org.apache.phoenix.util.TestUtil.A_VALUE;
+import static org.apache.phoenix.util.TestUtil.B_VALUE;
+import static org.apache.phoenix.util.TestUtil.C_VALUE;
+import static org.apache.phoenix.util.TestUtil.E_VALUE;
+import static org.apache.phoenix.util.TestUtil.ROW1;
+import static org.apache.phoenix.util.TestUtil.ROW2;
+import static org.apache.phoenix.util.TestUtil.ROW3;
+import static org.apache.phoenix.util.TestUtil.ROW4;
+import static org.apache.phoenix.util.TestUtil.ROW5;
+import static org.apache.phoenix.util.TestUtil.ROW6;
+import static org.apache.phoenix.util.TestUtil.ROW7;
+import static org.apache.phoenix.util.TestUtil.ROW8;
+import static org.apache.phoenix.util.TestUtil.ROW9;
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLFeatureNotSupportedException;
+import java.util.Properties;
+
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.junit.Test;
+
+public class DerivedTableIT extends BaseClientManagedTimeIT {
--- End diff --
One easy way would be to add these tests to QueryIT instead, though I like this separation too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11856917
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/compile/SubselectRewriter.java ---
@@ -0,0 +1,242 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.phoenix.compile;
+
+import java.sql.SQLException;
+import java.sql.SQLFeatureNotSupportedException;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.parse.AliasedNode;
+import org.apache.phoenix.parse.ColumnParseNode;
+import org.apache.phoenix.parse.DerivedTableNode;
+import org.apache.phoenix.parse.HintNode;
+import org.apache.phoenix.parse.LimitNode;
+import org.apache.phoenix.parse.OrderByNode;
+import org.apache.phoenix.parse.ParseNode;
+import org.apache.phoenix.parse.ParseNodeRewriter;
+import org.apache.phoenix.parse.SelectStatement;
+import org.apache.phoenix.parse.TableNode;
+import org.apache.phoenix.parse.TableWildcardParseNode;
+import org.apache.phoenix.parse.WildcardParseNode;
+import org.apache.phoenix.util.SchemaUtil;
+
+import com.google.common.collect.Lists;
+
+public class SubselectRewriter extends ParseNodeRewriter {
+ private final String tableAlias;
+ private final Map<String, ParseNode> aliasMap;
+
+ public static SelectStatement applyPostFilters(SelectStatement statement, List<ParseNode> postFilters, String subqueryAlias) throws SQLException {
+ if (postFilters.isEmpty())
+ return statement;
+
+ // TODO handle this from caller
+ if (statement.getLimit() != null || (statement.isAggregate() && statement.getGroupBy().isEmpty()))
--- End diff --
Yes, limit with literal numbers in both outer and inner queries can be handled. Please see "testDerivedTableWithLimit()".
Yes, agree. It would just require the compiler to handle derived tables. Right now we only support cases that can be handled by "pre-processing", i.e., rewriting; and let QueryCompiler to throw a SQLFeatureNotSupportedException if the from node is a derived table.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11837900
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/compile/SubselectRewriter.java ---
@@ -0,0 +1,242 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.phoenix.compile;
+
+import java.sql.SQLException;
+import java.sql.SQLFeatureNotSupportedException;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.phoenix.jdbc.PhoenixConnection;
+import org.apache.phoenix.parse.AliasedNode;
+import org.apache.phoenix.parse.ColumnParseNode;
+import org.apache.phoenix.parse.DerivedTableNode;
+import org.apache.phoenix.parse.HintNode;
+import org.apache.phoenix.parse.LimitNode;
+import org.apache.phoenix.parse.OrderByNode;
+import org.apache.phoenix.parse.ParseNode;
+import org.apache.phoenix.parse.ParseNodeRewriter;
+import org.apache.phoenix.parse.SelectStatement;
+import org.apache.phoenix.parse.TableNode;
+import org.apache.phoenix.parse.TableWildcardParseNode;
+import org.apache.phoenix.parse.WildcardParseNode;
+import org.apache.phoenix.util.SchemaUtil;
+
+import com.google.common.collect.Lists;
+
+public class SubselectRewriter extends ParseNodeRewriter {
+ private final String tableAlias;
+ private final Map<String, ParseNode> aliasMap;
+
+ public static SelectStatement applyPostFilters(SelectStatement statement, List<ParseNode> postFilters, String subqueryAlias) throws SQLException {
+ if (postFilters.isEmpty())
+ return statement;
+
+ // TODO handle this from caller
+ if (statement.getLimit() != null || (statement.isAggregate() && statement.getGroupBy().isEmpty()))
--- End diff --
Let's document these cases we're not currently handling. Is the limit one where the outer limit is less than the inner limit? Can we handle that?
The second case is an aggregate over a aggregate? We can't handle that, but it wouldn't be too difficult (if we do it client side in-memory).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11838095
--- Diff: phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java ---
@@ -0,0 +1,632 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.phoenix.end2end;
+
+import static org.apache.phoenix.util.TestUtil.A_VALUE;
+import static org.apache.phoenix.util.TestUtil.B_VALUE;
+import static org.apache.phoenix.util.TestUtil.C_VALUE;
+import static org.apache.phoenix.util.TestUtil.E_VALUE;
+import static org.apache.phoenix.util.TestUtil.ROW1;
+import static org.apache.phoenix.util.TestUtil.ROW2;
+import static org.apache.phoenix.util.TestUtil.ROW3;
+import static org.apache.phoenix.util.TestUtil.ROW4;
+import static org.apache.phoenix.util.TestUtil.ROW5;
+import static org.apache.phoenix.util.TestUtil.ROW6;
+import static org.apache.phoenix.util.TestUtil.ROW7;
+import static org.apache.phoenix.util.TestUtil.ROW8;
+import static org.apache.phoenix.util.TestUtil.ROW9;
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLFeatureNotSupportedException;
+import java.util.Properties;
+
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.junit.Test;
+
+public class DerivedTableIT extends BaseClientManagedTimeIT {
--- End diff --
Might be nice to parameterize this and test when an index may be used. Is that code path tested?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11838044
--- Diff: phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java ---
@@ -0,0 +1,632 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.phoenix.end2end;
+
+import static org.apache.phoenix.util.TestUtil.A_VALUE;
+import static org.apache.phoenix.util.TestUtil.B_VALUE;
+import static org.apache.phoenix.util.TestUtil.C_VALUE;
+import static org.apache.phoenix.util.TestUtil.E_VALUE;
+import static org.apache.phoenix.util.TestUtil.ROW1;
+import static org.apache.phoenix.util.TestUtil.ROW2;
+import static org.apache.phoenix.util.TestUtil.ROW3;
+import static org.apache.phoenix.util.TestUtil.ROW4;
+import static org.apache.phoenix.util.TestUtil.ROW5;
+import static org.apache.phoenix.util.TestUtil.ROW6;
+import static org.apache.phoenix.util.TestUtil.ROW7;
+import static org.apache.phoenix.util.TestUtil.ROW8;
+import static org.apache.phoenix.util.TestUtil.ROW9;
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLFeatureNotSupportedException;
+import java.util.Properties;
+
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.junit.Test;
+
+public class DerivedTableIT extends BaseClientManagedTimeIT {
--- End diff --
Nice tests!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11862579
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/compile/QueryCompiler.java ---
@@ -290,6 +292,9 @@ protected BasicQueryPlan compileSingleQuery(StatementContext context, SelectStat
ColumnResolver resolver = context.getResolver();
TableRef tableRef = context.getCurrentTable();
PTable table = tableRef.getTable();
+ if (table.getType() == PTableType.SUBQUERY)
+ throw new SQLFeatureNotSupportedException("Complex nested queries not supported.");
--- End diff --
Same comment here - can you add a code comment and JIRA for this? An example query that would produce this exception would be helpful too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue closed the pull request at:
https://github.com/apache/incubator-phoenix/pull/31
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-phoenix pull request: PHOENIX-136 Support derived tables
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/31#discussion_r11837911
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/compile/QueryCompiler.java ---
@@ -290,6 +292,9 @@ protected BasicQueryPlan compileSingleQuery(StatementContext context, SelectStat
ColumnResolver resolver = context.getResolver();
TableRef tableRef = context.getCurrentTable();
PTable table = tableRef.getTable();
+ if (table.getType() == PTableType.SUBQUERY)
+ throw new SQLFeatureNotSupportedException("Complex nested queries not supported.");
--- End diff --
Add a bit more docs, please. Is this a correlated subquery?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---