You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2022/08/11 00:07:00 UTC
Change in asterixdb[master]: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
From Glenn Galvizo <gg...@uci.edu>:
Glenn Galvizo has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
......................................................................
[WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
- user model changes: yes
- storage format changes: no
- interface changes: no
details:
- adding syntactic sugar rewrite to specify what fields to
EXCLUDE in a SELECT clause
Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
---
M asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.5.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.6.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.3.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.4.adm
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
A asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExceptRewriteSugarVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
22 files changed, 498 insertions(+), 21 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/17103/1
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
index dcd52a0..e04fee1 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
@@ -227,7 +227,7 @@
}
// Constructing select clause
SelectElement selectElement = new SelectElement(previousVarExpr);
- SelectClause selectClause = new SelectClause(selectElement, null, false);
+ SelectClause selectClause = new SelectClause(selectElement, null, null, false);
SelectBlock selectBlock = new SelectBlock(selectClause, fromClause,
whereClause != null ? Collections.singletonList(whereClause) : null, null, null);
SelectSetOperation selectSetOperation = new SelectSetOperation(new SetOperationInput(selectBlock, null), null);
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.1.ddl.sqlpp
new file mode 100644
index 0000000..3cb07d0
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.1.ddl.sqlpp
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+DROP DATAVERSE TinySocial IF EXISTS;
+CREATE DATAVERSE TinySocial;
+USE TinySocial;
+
+CREATE TYPE TwitterUserType AS {
+ `screen-name` : string,
+ lang : string,
+ friends_count : bigint,
+ statuses_count : bigint
+};
+
+CREATE TYPE TweetMessageType AS {
+ tweetid : string,
+ user : TwitterUserType,
+ `sender-location` : point?,
+ `send-time` : datetime,
+ `referred-topics` : {{string}},
+ `message-text` : string
+};
+
+CREATE DATASET TwitterUsers(TwitterUserType) PRIMARY KEY `screen-name`;
+CREATE DATASET TweetMessages(TweetMessageType) PRIMARY KEY tweetid;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.2.update.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.2.update.sqlpp
new file mode 100644
index 0000000..a60ef4d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.2.update.sqlpp
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE TinySocial;
+LOAD DATASET TwitterUsers USING localfs ((`path`=`asterix_nc1://data/tinysocial/twu.adm`),(`format`=`adm`));
+LOAD DATASET TweetMessages USING localfs ((`path`=`asterix_nc1://data/tinysocial/twm.adm`),(`format`=`adm`));
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.3.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.3.query.sqlpp
new file mode 100644
index 0000000..33e86f6
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.3.query.sqlpp
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+FROM TinySocial.TweetMessages TM
+
+ // SELECT-ELEMENT.
+SELECT VALUE TM
+
+ // Nested field that does not exist.
+EXCEPT user.does_not_exist,
+ // Nested field that does exist.
+ user.lang,
+ // Nested field with quotes.
+ user.`send-time`,
+ // Non-nested field that does not exist.
+ does_not_exist,
+ // Non-nested field that exists.
+ tweetid
+
+ORDER BY TM.tweetid;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.4.query.sqlpp
new file mode 100644
index 0000000..ff211af
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.4.query.sqlpp
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+FROM TinySocial.TweetMessages TM
+
+ // SELECT v.* (results in different AST to SELECT ELEMENT).
+SELECT TM.*
+
+ // Nested field that does not exist.
+EXCEPT user.does_not_exist,
+ // Nested field that does exist.
+ user.lang,
+ // Nested field with quotes.
+ user.`send-time`,
+ // Non-nested field that does not exist.
+ does_not_exist,
+ // Non-nested field that exists.
+ tweetid
+
+ORDER BY TM.tweetid;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.5.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.5.query.sqlpp
new file mode 100644
index 0000000..f0adab5
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.5.query.sqlpp
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+FROM TinySocial.TweetMessages TM,
+ TinySocial.TwitterUsers TU
+
+WHERE TM.user.`screen-name` = TU.`screen-name`
+
+ // SELECT * returns { "TM": ..., "TU": ... }
+SELECT *
+
+ // Identifier that exists in scope.
+EXCEPT TU,
+ // Nested identifier that exists in scope.
+ TM.user.`screen-name`,
+ // Identifier that does not exist in scope.
+ TI
+
+ORDER BY TM.tweetid;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.6.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.6.query.sqlpp
new file mode 100644
index 0000000..a113d26
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.6.query.sqlpp
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+FROM TinySocial.TweetMessages TM1
+SELECT VALUE TM1
+EXCEPT user.does_not_exist,
+ user.lang,
+ user.`send-time`,
+ does_not_exist
+
+UNION ALL
+
+FROM TinySocial.TweetMessages TM2
+SELECT VALUE TM2
+EXCEPT user.lang
+
+ORDER BY tweetid;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.3.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.3.adm
new file mode 100644
index 0000000..2358092
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.3.adm
@@ -0,0 +1,12 @@
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [47.44, 80.65], "send-time": "2008-04-26T10:10:00.000", "referred-topics": [ "t-mobile", "customization" ], "message-text": " love t-mobile its customization is good:)" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [29.15, 76.53], "send-time": "2008-01-26T10:10:00.000", "referred-topics": [ "verizon", "voice-clarity" ], "message-text": " hate verizon its voice-clarity is OMG:(" }
+{ "user": { "screen-name": "NilaMilliron_tw", "friends_count": 445, "statuses_count": 164, "name": "Nila Milliron", "followers_count": 22649 }, "sender-location": [37.59, 68.42], "send-time": "2008-03-09T10:10:00.000", "referred-topics": [ "iphone", "platform" ], "message-text": " can't stand iphone its platform is terrible" }
+{ "user": { "screen-name": "OliJackson_512", "friends_count": 445, "statuses_count": 164, "name": "Oli Jackson", "followers_count": 22649 }, "sender-location": [24.82, 94.63], "send-time": "2010-02-13T10:10:00.000", "referred-topics": [ "samsung", "voice-command" ], "message-text": " like samsung the voice-command is amazing:)" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [32.84, 67.14], "send-time": "2010-05-13T10:10:00.000", "referred-topics": [ "verizon", "shortcut-menu" ], "message-text": " like verizon its shortcut-menu is awesome:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [29.72, 75.8], "send-time": "2006-11-04T10:10:00.000", "referred-topics": [ "motorola", "speed" ], "message-text": " like motorola the speed is good:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [39.28, 70.48], "send-time": "2011-12-26T10:10:00.000", "referred-topics": [ "sprint", "voice-command" ], "message-text": " like sprint the voice-command is mind-blowing:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [40.09, 92.69], "send-time": "2006-08-04T10:10:00.000", "referred-topics": [ "motorola", "speed" ], "message-text": " can't stand motorola its speed is terrible:(" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [47.51, 83.99], "send-time": "2010-05-07T10:10:00.000", "referred-topics": [ "iphone", "voice-clarity" ], "message-text": " like iphone the voice-clarity is good:)" }
+{ "user": { "screen-name": "ChangEwing_573", "friends_count": 182, "statuses_count": 394, "name": "Chang Ewing", "followers_count": 32136 }, "sender-location": [36.21, 72.6], "send-time": "2011-08-25T10:10:00.000", "referred-topics": [ "samsung", "platform" ], "message-text": " like samsung the platform is good" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [46.05, 93.34], "send-time": "2005-10-14T10:10:00.000", "referred-topics": [ "t-mobile", "shortcut-menu" ], "message-text": " like t-mobile the shortcut-menu is awesome:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [36.86, 74.62], "send-time": "2012-07-21T10:10:00.000", "referred-topics": [ "verizon", "voicemail-service" ], "message-text": " love verizon its voicemail-service is awesome" }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.4.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.4.adm
new file mode 100644
index 0000000..2358092
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.4.adm
@@ -0,0 +1,12 @@
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [47.44, 80.65], "send-time": "2008-04-26T10:10:00.000", "referred-topics": [ "t-mobile", "customization" ], "message-text": " love t-mobile its customization is good:)" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [29.15, 76.53], "send-time": "2008-01-26T10:10:00.000", "referred-topics": [ "verizon", "voice-clarity" ], "message-text": " hate verizon its voice-clarity is OMG:(" }
+{ "user": { "screen-name": "NilaMilliron_tw", "friends_count": 445, "statuses_count": 164, "name": "Nila Milliron", "followers_count": 22649 }, "sender-location": [37.59, 68.42], "send-time": "2008-03-09T10:10:00.000", "referred-topics": [ "iphone", "platform" ], "message-text": " can't stand iphone its platform is terrible" }
+{ "user": { "screen-name": "OliJackson_512", "friends_count": 445, "statuses_count": 164, "name": "Oli Jackson", "followers_count": 22649 }, "sender-location": [24.82, 94.63], "send-time": "2010-02-13T10:10:00.000", "referred-topics": [ "samsung", "voice-command" ], "message-text": " like samsung the voice-command is amazing:)" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [32.84, 67.14], "send-time": "2010-05-13T10:10:00.000", "referred-topics": [ "verizon", "shortcut-menu" ], "message-text": " like verizon its shortcut-menu is awesome:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [29.72, 75.8], "send-time": "2006-11-04T10:10:00.000", "referred-topics": [ "motorola", "speed" ], "message-text": " like motorola the speed is good:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [39.28, 70.48], "send-time": "2011-12-26T10:10:00.000", "referred-topics": [ "sprint", "voice-command" ], "message-text": " like sprint the voice-command is mind-blowing:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [40.09, 92.69], "send-time": "2006-08-04T10:10:00.000", "referred-topics": [ "motorola", "speed" ], "message-text": " can't stand motorola its speed is terrible:(" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [47.51, 83.99], "send-time": "2010-05-07T10:10:00.000", "referred-topics": [ "iphone", "voice-clarity" ], "message-text": " like iphone the voice-clarity is good:)" }
+{ "user": { "screen-name": "ChangEwing_573", "friends_count": 182, "statuses_count": 394, "name": "Chang Ewing", "followers_count": 32136 }, "sender-location": [36.21, 72.6], "send-time": "2011-08-25T10:10:00.000", "referred-topics": [ "samsung", "platform" ], "message-text": " like samsung the platform is good" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [46.05, 93.34], "send-time": "2005-10-14T10:10:00.000", "referred-topics": [ "t-mobile", "shortcut-menu" ], "message-text": " like t-mobile the shortcut-menu is awesome:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [36.86, 74.62], "send-time": "2012-07-21T10:10:00.000", "referred-topics": [ "verizon", "voicemail-service" ], "message-text": " love verizon its voicemail-service is awesome" }
\ No newline at end of file
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
index ea6bac9..7ef8982 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
@@ -19,7 +19,11 @@
package org.apache.asterix.lang.sqlpp.clause;
+import java.util.ArrayList;
+import java.util.List;
import java.util.Objects;
+import java.util.function.Function;
+import java.util.stream.Collectors;
import org.apache.asterix.common.exceptions.CompilationException;
import org.apache.asterix.lang.common.base.AbstractClause;
@@ -27,15 +31,19 @@
import org.apache.asterix.lang.sqlpp.visitor.base.ISqlppVisitor;
public class SelectClause extends AbstractClause {
-
+ private final List<List<String>> fieldExclusions = new ArrayList<>();
private SelectElement selectElement;
private SelectRegular selectRegular;
private boolean distinct;
- public SelectClause(SelectElement selectElement, SelectRegular selectRegular, boolean distinct) {
+ public SelectClause(SelectElement selectElement, SelectRegular selectRegular, List<List<String>> fieldExclusions,
+ boolean distinct) {
this.selectElement = selectElement;
this.selectRegular = selectRegular;
this.distinct = distinct;
+ if (fieldExclusions != null) {
+ this.fieldExclusions.addAll(fieldExclusions);
+ }
}
@Override
@@ -82,15 +90,25 @@
this.distinct = distinct;
}
+ public List<List<String>> getFieldExclusions() {
+ return fieldExclusions;
+ }
+
@Override
public String toString() {
- return "select " + (distinct ? "distinct " : "")
- + (selectElement() ? "element " + selectElement : String.valueOf(selectRegular));
+ String distinctString = distinct ? "distinct " : "";
+ String valueString = selectElement() ? ("element " + selectElement) : String.valueOf(selectRegular);
+ String exceptString = "";
+ if (!fieldExclusions.isEmpty()) {
+ final Function<List<String>, String> fieldBuilder = f -> String.join(".", f);
+ exceptString = " except " + fieldExclusions.stream().map(fieldBuilder).collect(Collectors.joining(", "));
+ }
+ return String.format("select %s%s%s", distinctString, valueString, exceptString);
}
@Override
public int hashCode() {
- return Objects.hash(distinct, selectElement, selectRegular);
+ return Objects.hash(distinct, selectElement, selectRegular, fieldExclusions);
}
@Override
@@ -103,6 +121,7 @@
}
SelectClause target = (SelectClause) object;
return distinct == target.distinct && Objects.equals(selectElement, target.selectElement)
- && Objects.equals(selectRegular, target.selectRegular);
+ && Objects.equals(selectRegular, target.selectRegular)
+ && Objects.equals(fieldExclusions, target.fieldExclusions);
}
}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
index 4352210..71d88fa 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
@@ -130,6 +130,9 @@
// Rewrites RIGHT OUTER JOINs into LEFT OUTER JOINs if possible
rewriteRightJoins();
+
+ // Rewrite SELECT EXCEPT to use OBJECT_REMOVE_FIELDS.
+ rewriteSelectExceptSugar();
}
static Expression castViewBodyAsType(LangRewritingContext context, Expression bodyExpr, IAType itemType,
@@ -218,7 +221,7 @@
RecordConstructor recordConstr = new RecordConstructor(projections);
recordConstr.setSourceLocation(sourceLoc);
- SelectClause selectClause = new SelectClause(new SelectElement(recordConstr), null, false);
+ SelectClause selectClause = new SelectClause(new SelectElement(recordConstr), null, null, false);
selectClause.setSourceLocation(sourceLoc);
SelectBlock selectBlock = new SelectBlock(selectClause, fromClause, letWhereClauseList, null, null);
selectBlock.setSourceLocation(sourceLoc);
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
index 2f4fcc8..93360a9 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
@@ -58,6 +58,7 @@
import org.apache.asterix.lang.sqlpp.rewrites.visitor.InlineColumnAliasVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.InlineWithExpressionVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.OperatorExpressionVisitor;
+import org.apache.asterix.lang.sqlpp.rewrites.visitor.SelectExceptRewriteSugarVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.SetOperationVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.SqlCompatRewriteVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.SqlppCaseAggregateExtractionVisitor;
@@ -210,6 +211,9 @@
// and external UDFs, therefore need to inline SQL++ UDFs to check the deterministic property.
inlineWithExpressions();
+ // Rewrite SELECT EXCEPT to use OBJECT_REMOVE_FIELDS.
+ rewriteSelectExceptSugar();
+
// Sets the var counter of the query.
topStatement.setVarCounter(context.getVarCounter().get());
}
@@ -350,6 +354,11 @@
}
}
+ protected void rewriteSelectExceptSugar() throws CompilationException {
+ SelectExceptRewriteSugarVisitor selectExceptRewriteSugarVisitor = new SelectExceptRewriteSugarVisitor(context);
+ rewriteTopExpr(selectExceptRewriteSugarVisitor, null);
+ }
+
private <R, T> R rewriteTopExpr(ILangVisitor<R, T> visitor, T arg) throws CompilationException {
R result = topStatement.accept(visitor, arg);
logExpression(">>>> AST After", visitor.getClass().getSimpleName());
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExceptRewriteSugarVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExceptRewriteSugarVisitor.java
new file mode 100644
index 0000000..af8fb6e
--- /dev/null
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExceptRewriteSugarVisitor.java
@@ -0,0 +1,195 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.lang.sqlpp.rewrites.visitor;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Deque;
+import java.util.List;
+import java.util.stream.Collectors;
+
+import org.apache.asterix.common.exceptions.CompilationException;
+import org.apache.asterix.common.functions.FunctionSignature;
+import org.apache.asterix.lang.common.base.Expression;
+import org.apache.asterix.lang.common.base.ILangExpression;
+import org.apache.asterix.lang.common.expression.CallExpr;
+import org.apache.asterix.lang.common.expression.ListConstructor;
+import org.apache.asterix.lang.common.expression.LiteralExpr;
+import org.apache.asterix.lang.common.expression.VariableExpr;
+import org.apache.asterix.lang.common.literal.StringLiteral;
+import org.apache.asterix.lang.common.rewrites.LangRewritingContext;
+import org.apache.asterix.lang.common.struct.VarIdentifier;
+import org.apache.asterix.lang.sqlpp.clause.FromClause;
+import org.apache.asterix.lang.sqlpp.clause.FromTerm;
+import org.apache.asterix.lang.sqlpp.clause.SelectBlock;
+import org.apache.asterix.lang.sqlpp.clause.SelectClause;
+import org.apache.asterix.lang.sqlpp.clause.SelectElement;
+import org.apache.asterix.lang.sqlpp.clause.SelectSetOperation;
+import org.apache.asterix.lang.sqlpp.expression.SelectExpression;
+import org.apache.asterix.lang.sqlpp.struct.SetOperationInput;
+import org.apache.asterix.lang.sqlpp.struct.SetOperationRight;
+import org.apache.asterix.lang.sqlpp.visitor.base.AbstractSqlppExpressionScopingVisitor;
+import org.apache.asterix.om.functions.BuiltinFunctions;
+
+/**
+ * Rewrites the exclusion list of a SELECT clause into OBJECT_REMOVE_FIELDS function applications.
+ * Input:
+ * <pre>
+ * FROM ...
+ * WHERE ...
+ * SELECT * EXCEPT a, b.f
+ * ORDER BY c
+ * </pre>
+ * Output:
+ * <pre>
+ * FROM ( FROM ...
+ * WHERE ...
+ * SELECT *
+ * ORDER BY c ) TMP_1
+ * SELECT VALUE OBJECT_REMOVE_FIELDS(TMP_1, [ "a", [ "b", "f" ]])
+ * </pre>
+ */
+public class SelectExceptRewriteSugarVisitor extends AbstractSqlppExpressionScopingVisitor {
+ private final Deque<SelectExpression> selectExpressionStack = new ArrayDeque<>();
+
+ public SelectExceptRewriteSugarVisitor(LangRewritingContext langRewritingContext) {
+ super(langRewritingContext);
+ }
+
+ /**
+ * We will only return a **new** SELECT-EXPR if we have a SELECT-EXPR with output modifiers, no set operations, and
+ * with a SELECT-BLOCK whose SELECT-CLAUSE contains field exclusions. For all other cases, we will localize our
+ * changes to the affected SELECT-BLOCK.
+ */
+ @Override
+ public Expression visit(SelectExpression selectExpression, ILangExpression arg) throws CompilationException {
+ super.visit(selectExpression, arg);
+ return selectExpressionStack.removeLast();
+ }
+
+ @Override
+ public Expression visit(SelectSetOperation selectSetOperation, ILangExpression arg) throws CompilationException {
+ SelectExpression selectExpression = (SelectExpression) arg;
+ boolean hasOutputModifiers = selectExpression.hasLimit() || selectExpression.hasOrderby();
+ boolean hasRightInputs = selectSetOperation.hasRightInputs();
+ if (hasOutputModifiers && !hasRightInputs) {
+ selectExpressionStack.addLast(wrapSelectExpression(selectExpression));
+ } else {
+ mutateAndWrapSelectBlock(selectSetOperation.getLeftInput(), selectExpression);
+ for (SetOperationRight rightInput : selectSetOperation.getRightInputs()) {
+ mutateAndWrapSelectBlock(rightInput.getSetOperationRightInput(), selectExpression);
+ }
+ selectExpressionStack.addLast(selectExpression);
+ }
+ return null;
+ }
+
+ private SelectExpression wrapSelectExpression(SelectExpression selectExpression) {
+ SelectSetOperation selectSetOperation = selectExpression.getSelectSetOperation();
+ SetOperationInput setOperationInput = selectSetOperation.getLeftInput();
+ if (!setOperationInput.selectBlock()) {
+ return selectExpression;
+ }
+
+ SelectBlock innerSelectBlock = selectExpression.getSelectSetOperation().getLeftInput().getSelectBlock();
+ SelectClause originalSelectClause = innerSelectBlock.getSelectClause();
+ if (originalSelectClause.getFieldExclusions().isEmpty()) {
+ return selectExpression;
+ }
+
+ // Nest our original SELECT-EXPR.
+ SetOperationInput innerSetOpInput = new SetOperationInput(null, selectExpression);
+ SelectSetOperation innerSelectSetOp = new SelectSetOperation(innerSetOpInput, null);
+ SelectExpression innerSelectExpr = new SelectExpression(null, innerSelectSetOp, null, null, true);
+
+ // Build a new SELECT-EXPR.
+ VarIdentifier fromTermVariable = context.newVariable();
+ VariableExpr fromTermVariableExpr = new VariableExpr(fromTermVariable);
+ SelectClause selectClause = buildSelectClause(originalSelectClause, fromTermVariable);
+ FromTerm fromTerm = new FromTerm(innerSelectExpr, fromTermVariableExpr, null, null);
+ FromClause fromClause = new FromClause(List.of(fromTerm));
+ SelectBlock selectBlock = new SelectBlock(selectClause, fromClause, null, null, null);
+ SetOperationInput newSetOpInput = new SetOperationInput(selectBlock, null);
+ SelectSetOperation newSelectSetOperation = new SelectSetOperation(newSetOpInput, null);
+ SelectExpression newSelectExpression = new SelectExpression(null, newSelectSetOperation, null, null, true);
+ newSelectExpression.setSourceLocation(selectExpression.getSourceLocation());
+ return newSelectExpression;
+ }
+
+ private void mutateAndWrapSelectBlock(SetOperationInput setOperationInput, ILangExpression arg)
+ throws CompilationException {
+ Expression visitResult = setOperationInput.accept(this, arg);
+ if (setOperationInput.subquery()) {
+ setOperationInput.setSubquery((SelectExpression) visitResult);
+ }
+
+ // Only proceed if we have fields in our EXCEPT list.
+ SelectBlock innerSelectBlock = setOperationInput.getSelectBlock();
+ SelectClause originalSelectClause = innerSelectBlock.getSelectClause();
+ if (originalSelectClause.getFieldExclusions().isEmpty()) {
+ return;
+ }
+
+ // Nest our original SELECT-BLOCK.
+ SetOperationInput innerSetOpInput = new SetOperationInput(innerSelectBlock, null);
+ SelectSetOperation innerSelectSetOp = new SelectSetOperation(innerSetOpInput, null);
+ SelectExpression innerSelectExpr = new SelectExpression(null, innerSelectSetOp, null, null, true);
+
+ // Build a new SELECT-BLOCK.
+ VarIdentifier fromTermVariable = context.newVariable();
+ VariableExpr fromTermVariableExpr = new VariableExpr(fromTermVariable);
+ SelectClause selectClause = buildSelectClause(originalSelectClause, fromTermVariable);
+ FromTerm fromTerm = new FromTerm(innerSelectExpr, fromTermVariableExpr, null, null);
+ FromClause fromClause = new FromClause(List.of(fromTerm));
+ SelectBlock selectBlock = new SelectBlock(selectClause, fromClause, null, null, null);
+ setOperationInput.setSelectBlock(selectBlock);
+ }
+
+ private SelectClause buildSelectClause(SelectClause originalSelectClause, VarIdentifier iterationVariable) {
+ // Convert our list of identifiers into a list of literals representing field names.
+ ListConstructor listConstructor = new ListConstructor();
+ listConstructor.setType(ListConstructor.Type.ORDERED_LIST_CONSTRUCTOR);
+ listConstructor.setExprList(new ArrayList<>());
+ for (List<String> nestedField : originalSelectClause.getFieldExclusions()) {
+ if (nestedField.size() == 1) {
+ // For non-nested fields, we do not wrap our name in a list.
+ listConstructor.getExprList().add(new LiteralExpr(new StringLiteral(nestedField.get(0))));
+ } else {
+ // Otherwise, build a list to insert into our list.
+ ListConstructor nestedFieldList = new ListConstructor();
+ nestedFieldList.setType(ListConstructor.Type.ORDERED_LIST_CONSTRUCTOR);
+ nestedFieldList.setExprList(nestedField.stream().map(f -> new LiteralExpr(new StringLiteral(f)))
+ .collect(Collectors.toList()));
+ listConstructor.getExprList().add(nestedFieldList);
+ }
+ }
+ List<Expression> objectRemoveFieldsArguments = new ArrayList<>();
+ objectRemoveFieldsArguments.add(new VariableExpr(iterationVariable));
+ objectRemoveFieldsArguments.add(listConstructor);
+
+ // Create the call to OBJECT_REMOVE_FIELDS.
+ FunctionSignature functionSignature = new FunctionSignature(BuiltinFunctions.REMOVE_FIELDS);
+ CallExpr callExpr = new CallExpr(functionSignature, objectRemoveFieldsArguments);
+ SelectElement selectElement = new SelectElement(callExpr);
+ boolean distinct = originalSelectClause.distinct();
+ SelectClause selectClause = new SelectClause(selectElement, null, null, distinct);
+ selectClause.setSourceLocation(originalSelectClause.getSourceLocation());
+ return selectClause;
+ }
+}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java
index 1266d9a..2ff4808 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java
@@ -112,7 +112,7 @@
newFromClause.setSourceLocation(sourceLoc);
Expression valueExpr = valueExprTransformer != null
? valueExprTransformer.apply(newBindingVar, valueExprTransformerArg) : newBindingVar;
- SelectClause selectClause = new SelectClause(new SelectElement(valueExpr), null, distinct);
+ SelectClause selectClause = new SelectClause(new SelectElement(valueExpr), null, null, distinct);
selectClause.setSourceLocation(sourceLoc);
SelectBlock selectBlock = new SelectBlock(selectClause, newFromClause, null, null, null);
selectBlock.setSourceLocation(sourceLoc);
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java
index f93e843..a2eed4f 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java
@@ -159,7 +159,7 @@
SelectElement selectElement = new SelectElement(newExpr);
selectElement.setSourceLocation(sourceLoc);
- SelectClause selectClause = new SelectClause(selectElement, null, false);
+ SelectClause selectClause = new SelectClause(selectElement, null, null, false);
selectClause.setSourceLocation(sourceLoc);
// Construct the select expression.
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
index b98151b..4ebd50a5 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
@@ -416,7 +416,7 @@
}
RecordConstructor rc1 = new RecordConstructor(fb1List);
rc1.setSourceLocation(sourceLoc);
- SelectClause selectClause1 = new SelectClause(new SelectElement(rc1), null, false);
+ SelectClause selectClause1 = new SelectClause(new SelectElement(rc1), null, null, false);
selectClause1.setSourceLocation(sourceLoc);
SelectBlock newRightSelectBlock = new SelectBlock(selectClause1, fromClause1, null, null, null);
newRightSelectBlock.setSourceLocation(sourceLoc);
@@ -650,7 +650,7 @@
fa.setSourceLocation(sourceLoc);
SelectElement sv1 = new SelectElement(fa);
sv1.setSourceLocation(sourceLoc);
- SelectClause sc1 = new SelectClause(sv1, null, false);
+ SelectClause sc1 = new SelectClause(sv1, null, null, false);
sc1.setSourceLocation(sourceLoc);
SelectBlock sb1 = new SelectBlock(sc1, fc1, null, null, null);
sv1.setSourceLocation(sourceLoc);
@@ -691,7 +691,7 @@
v1Ref2.setSourceLocation(sourceLoc);
SelectElement sv1 = new SelectElement(v1Ref2);
sv1.setSourceLocation(sourceLoc);
- SelectClause sc1 = new SelectClause(sv1, null, false);
+ SelectClause sc1 = new SelectClause(sv1, null, null, false);
sc1.setSourceLocation(sourceLoc);
SelectBlock sb1 = new SelectBlock(sc1, fc1, null, null, null);
sv1.setSourceLocation(sourceLoc);
@@ -743,7 +743,7 @@
SelectElement sv2 = new SelectElement(iaExpr);
sv2.setSourceLocation(sourceLoc);
- SelectClause sc2 = new SelectClause(sv2, null, false);
+ SelectClause sc2 = new SelectClause(sv2, null, null, false);
sc2.setSourceLocation(sourceLoc);
List<AbstractClause> sb2Clauses = new ArrayList<>(1);
sb2Clauses.add(lc2);
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java
index 1259b64..5a565be 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java
@@ -411,7 +411,8 @@
.add(new Projection(Projection.Kind.NAMED_EXPR, selectElement.getExpression(), mainProjectionName));
SelectRegular newSelectRegular = new SelectRegular(projectionList);
newSelectRegular.setSourceLocation(selectElement.getSourceLocation());
- SelectClause newSelectClause = new SelectClause(null, newSelectRegular, selectClause.distinct());
+ SelectClause newSelectClause =
+ new SelectClause(null, newSelectRegular, selectClause.getFieldExclusions(), selectClause.distinct());
newSelectClause.setSourceLocation(selectClause.getSourceLocation());
return newSelectClause;
}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java
index 26c6915..e6042f4 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java
@@ -354,7 +354,7 @@
newFromTermList.add(newFromTerm);
FromClause newFromClause = new FromClause(newFromTermList);
newFromClause.setSourceLocation(sourceLoc);
- SelectClause newSelectClause = new SelectClause(null, new SelectRegular(projectList), false);
+ SelectClause newSelectClause = new SelectClause(null, new SelectRegular(projectList), null, false);
newSelectClause.setSourceLocation(sourceLoc);
SelectBlock newSelectBlock = new SelectBlock(newSelectClause, newFromClause, null, null, null);
newSelectBlock.setSourceLocation(sourceLoc);
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
index d74d047..60d93b4 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
@@ -193,7 +193,8 @@
if (selectClause.selectRegular()) {
selectRegular = (SelectRegular) selectClause.getSelectRegular().accept(this, arg);
}
- SelectClause copy = new SelectClause(selectElement, selectRegular, selectClause.distinct());
+ List<List<String>> fieldExclusions = selectClause.getFieldExclusions();
+ SelectClause copy = new SelectClause(selectElement, selectRegular, fieldExclusions, selectClause.distinct());
copy.setSourceLocation(selectClause.getSourceLocation());
return copy;
}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
index ce0d0a1..63c89f6 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
@@ -263,13 +263,15 @@
if (selectClause.selectElement()) {
Pair<ILangExpression, VariableSubstitutionEnvironment> newSelectElement =
selectClause.getSelectElement().accept(this, env);
- SelectClause newSelectClause = new SelectClause((SelectElement) newSelectElement.first, null, distinct);
+ SelectClause newSelectClause = new SelectClause((SelectElement) newSelectElement.first, null,
+ selectClause.getFieldExclusions(), distinct);
newSelectClause.setSourceLocation(selectClause.getSourceLocation());
return new Pair<>(newSelectClause, newSelectElement.second);
} else {
Pair<ILangExpression, VariableSubstitutionEnvironment> newSelectRegular =
selectClause.getSelectRegular().accept(this, env);
- SelectClause newSelectClause = new SelectClause(null, (SelectRegular) newSelectRegular.first, distinct);
+ SelectClause newSelectClause = new SelectClause(null, (SelectRegular) newSelectRegular.first,
+ selectClause.getFieldExclusions(), distinct);
newSelectClause.setSourceLocation(selectClause.getSourceLocation());
return new Pair<>(newSelectClause, newSelectRegular.second);
}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java
index 7e7b654..c313f06 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java
@@ -89,7 +89,7 @@
returnExpr.setSourceLocation(var.getSourceLocation());
SelectElement selectElement = new SelectElement(returnExpr);
selectElement.setSourceLocation(deleteStmt.getSourceLocation());
- SelectClause selectClause = new SelectClause(selectElement, null, false);
+ SelectClause selectClause = new SelectClause(selectElement, null, null, false);
selectClause.setSourceLocation(deleteStmt.getSourceLocation());
// Construct the select expression.
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
index a37b9cc..52646c7 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
+++ b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
@@ -4724,6 +4724,9 @@
Token startToken = null;
SelectRegular selectRegular = null;
SelectElement selectElement = null;
+ List<List<String>> fieldExclusions = new ArrayList<List<String>>();
+ List<String> nestedField = new ArrayList<String>();
+ String identifier;
boolean distinct = false;
}
{
@@ -4733,6 +4736,23 @@
|
selectElement = SelectElement()
)?
+ (
+ <EXCEPT>
+ identifier = Identifier() { nestedField.add(identifier); }
+ ( <DOT> identifier = Identifier() { nestedField.add(identifier); } )*
+ {
+ fieldExclusions.add(nestedField);
+ nestedField = new ArrayList<String>();
+ }
+ (
+ <COMMA> identifier = Identifier() { nestedField.add(identifier); }
+ ( <DOT> identifier = Identifier() { nestedField.add(identifier); } )*
+ {
+ fieldExclusions.add(nestedField);
+ nestedField = new ArrayList<String>();
+ }
+ )*
+ )?
{
SourceLocation sourceLoc = getSourceLocation(startToken);
if (selectRegular == null && selectElement == null){
@@ -4743,7 +4763,7 @@
selectRegular = new SelectRegular(projections);
selectRegular.setSourceLocation(sourceLoc);
}
- SelectClause selectClause = new SelectClause(selectElement, selectRegular, distinct);
+ SelectClause selectClause = new SelectClause(selectElement, selectRegular, fieldExclusions, distinct);
selectClause.setSourceLocation(sourceLoc);
return selectClause;
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-MessageType: newchange
Change in asterixdb[master]: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Ali Alsuliman <al...@gmail.com>:
Attention is currently required from: Glenn Galvizo, Till Westmann.
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
......................................................................
Patch Set 15: Code-Review+2
(4 comments)
Patchset:
PS15:
few more tests to consider:
1. negative tests for cases where exclude does not make sense/illegal, e.g. with SELECT VALUE.
2. tests where a user wants to actually select a field called EXCLUDE.
3. a full-blown query that has all the possible clauses (e.g. including LET after GROUP-BY)
File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-exclude/complex-exclude/complex-exclude.3.query.sqlpp:
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103/comment/43db283d_5ed39506
PS15, Line 22: SELECT U.*
we can have another test that also tries to exclude from here.
File asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java:
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103/comment/355b9822_87b3e62f
PS15, Line 121: except
should be exclude now
File asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExcludeRewriteSugarVisitor.java:
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103/comment/ec45fca7_5e162afb
PS15, Line 154: assert setOperationInput != null;
we don't normally do "assert". if this should never happen, we should fail with an illegal state exception.
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 15
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@couchbase.com>
Gerrit-Attention: Glenn Galvizo <gg...@uci.edu>
Gerrit-Attention: Till Westmann <ti...@couchbase.com>
Gerrit-Comment-Date: Mon, 24 Oct 2022 14:46:24 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
Change in asterixdb[master]: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:
Glenn Galvizo has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
......................................................................
[WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
- user model changes: yes
- storage format changes: no
- interface changes: no
details:
- adding syntactic sugar rewrite to specify what fields to
EXCLUDE in a SELECT clause
Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
---
M asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.5.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.6.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.3.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.4.adm
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
A asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExceptRewriteSugarVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
22 files changed, 498 insertions(+), 21 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/17103/1
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
index dcd52a0..e04fee1 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
@@ -227,7 +227,7 @@
}
// Constructing select clause
SelectElement selectElement = new SelectElement(previousVarExpr);
- SelectClause selectClause = new SelectClause(selectElement, null, false);
+ SelectClause selectClause = new SelectClause(selectElement, null, null, false);
SelectBlock selectBlock = new SelectBlock(selectClause, fromClause,
whereClause != null ? Collections.singletonList(whereClause) : null, null, null);
SelectSetOperation selectSetOperation = new SelectSetOperation(new SetOperationInput(selectBlock, null), null);
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.1.ddl.sqlpp
new file mode 100644
index 0000000..3cb07d0
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.1.ddl.sqlpp
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+DROP DATAVERSE TinySocial IF EXISTS;
+CREATE DATAVERSE TinySocial;
+USE TinySocial;
+
+CREATE TYPE TwitterUserType AS {
+ `screen-name` : string,
+ lang : string,
+ friends_count : bigint,
+ statuses_count : bigint
+};
+
+CREATE TYPE TweetMessageType AS {
+ tweetid : string,
+ user : TwitterUserType,
+ `sender-location` : point?,
+ `send-time` : datetime,
+ `referred-topics` : {{string}},
+ `message-text` : string
+};
+
+CREATE DATASET TwitterUsers(TwitterUserType) PRIMARY KEY `screen-name`;
+CREATE DATASET TweetMessages(TweetMessageType) PRIMARY KEY tweetid;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.2.update.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.2.update.sqlpp
new file mode 100644
index 0000000..a60ef4d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.2.update.sqlpp
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE TinySocial;
+LOAD DATASET TwitterUsers USING localfs ((`path`=`asterix_nc1://data/tinysocial/twu.adm`),(`format`=`adm`));
+LOAD DATASET TweetMessages USING localfs ((`path`=`asterix_nc1://data/tinysocial/twm.adm`),(`format`=`adm`));
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.3.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.3.query.sqlpp
new file mode 100644
index 0000000..33e86f6
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.3.query.sqlpp
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+FROM TinySocial.TweetMessages TM
+
+ // SELECT-ELEMENT.
+SELECT VALUE TM
+
+ // Nested field that does not exist.
+EXCEPT user.does_not_exist,
+ // Nested field that does exist.
+ user.lang,
+ // Nested field with quotes.
+ user.`send-time`,
+ // Non-nested field that does not exist.
+ does_not_exist,
+ // Non-nested field that exists.
+ tweetid
+
+ORDER BY TM.tweetid;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.4.query.sqlpp
new file mode 100644
index 0000000..ff211af
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.4.query.sqlpp
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+FROM TinySocial.TweetMessages TM
+
+ // SELECT v.* (results in different AST to SELECT ELEMENT).
+SELECT TM.*
+
+ // Nested field that does not exist.
+EXCEPT user.does_not_exist,
+ // Nested field that does exist.
+ user.lang,
+ // Nested field with quotes.
+ user.`send-time`,
+ // Non-nested field that does not exist.
+ does_not_exist,
+ // Non-nested field that exists.
+ tweetid
+
+ORDER BY TM.tweetid;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.5.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.5.query.sqlpp
new file mode 100644
index 0000000..f0adab5
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.5.query.sqlpp
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+FROM TinySocial.TweetMessages TM,
+ TinySocial.TwitterUsers TU
+
+WHERE TM.user.`screen-name` = TU.`screen-name`
+
+ // SELECT * returns { "TM": ..., "TU": ... }
+SELECT *
+
+ // Identifier that exists in scope.
+EXCEPT TU,
+ // Nested identifier that exists in scope.
+ TM.user.`screen-name`,
+ // Identifier that does not exist in scope.
+ TI
+
+ORDER BY TM.tweetid;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.6.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.6.query.sqlpp
new file mode 100644
index 0000000..a113d26
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-except/select-except.6.query.sqlpp
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+FROM TinySocial.TweetMessages TM1
+SELECT VALUE TM1
+EXCEPT user.does_not_exist,
+ user.lang,
+ user.`send-time`,
+ does_not_exist
+
+UNION ALL
+
+FROM TinySocial.TweetMessages TM2
+SELECT VALUE TM2
+EXCEPT user.lang
+
+ORDER BY tweetid;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.3.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.3.adm
new file mode 100644
index 0000000..2358092
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.3.adm
@@ -0,0 +1,12 @@
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [47.44, 80.65], "send-time": "2008-04-26T10:10:00.000", "referred-topics": [ "t-mobile", "customization" ], "message-text": " love t-mobile its customization is good:)" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [29.15, 76.53], "send-time": "2008-01-26T10:10:00.000", "referred-topics": [ "verizon", "voice-clarity" ], "message-text": " hate verizon its voice-clarity is OMG:(" }
+{ "user": { "screen-name": "NilaMilliron_tw", "friends_count": 445, "statuses_count": 164, "name": "Nila Milliron", "followers_count": 22649 }, "sender-location": [37.59, 68.42], "send-time": "2008-03-09T10:10:00.000", "referred-topics": [ "iphone", "platform" ], "message-text": " can't stand iphone its platform is terrible" }
+{ "user": { "screen-name": "OliJackson_512", "friends_count": 445, "statuses_count": 164, "name": "Oli Jackson", "followers_count": 22649 }, "sender-location": [24.82, 94.63], "send-time": "2010-02-13T10:10:00.000", "referred-topics": [ "samsung", "voice-command" ], "message-text": " like samsung the voice-command is amazing:)" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [32.84, 67.14], "send-time": "2010-05-13T10:10:00.000", "referred-topics": [ "verizon", "shortcut-menu" ], "message-text": " like verizon its shortcut-menu is awesome:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [29.72, 75.8], "send-time": "2006-11-04T10:10:00.000", "referred-topics": [ "motorola", "speed" ], "message-text": " like motorola the speed is good:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [39.28, 70.48], "send-time": "2011-12-26T10:10:00.000", "referred-topics": [ "sprint", "voice-command" ], "message-text": " like sprint the voice-command is mind-blowing:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [40.09, 92.69], "send-time": "2006-08-04T10:10:00.000", "referred-topics": [ "motorola", "speed" ], "message-text": " can't stand motorola its speed is terrible:(" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [47.51, 83.99], "send-time": "2010-05-07T10:10:00.000", "referred-topics": [ "iphone", "voice-clarity" ], "message-text": " like iphone the voice-clarity is good:)" }
+{ "user": { "screen-name": "ChangEwing_573", "friends_count": 182, "statuses_count": 394, "name": "Chang Ewing", "followers_count": 32136 }, "sender-location": [36.21, 72.6], "send-time": "2011-08-25T10:10:00.000", "referred-topics": [ "samsung", "platform" ], "message-text": " like samsung the platform is good" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [46.05, 93.34], "send-time": "2005-10-14T10:10:00.000", "referred-topics": [ "t-mobile", "shortcut-menu" ], "message-text": " like t-mobile the shortcut-menu is awesome:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [36.86, 74.62], "send-time": "2012-07-21T10:10:00.000", "referred-topics": [ "verizon", "voicemail-service" ], "message-text": " love verizon its voicemail-service is awesome" }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.4.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.4.adm
new file mode 100644
index 0000000..2358092
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/select-except/select-except.4.adm
@@ -0,0 +1,12 @@
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [47.44, 80.65], "send-time": "2008-04-26T10:10:00.000", "referred-topics": [ "t-mobile", "customization" ], "message-text": " love t-mobile its customization is good:)" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [29.15, 76.53], "send-time": "2008-01-26T10:10:00.000", "referred-topics": [ "verizon", "voice-clarity" ], "message-text": " hate verizon its voice-clarity is OMG:(" }
+{ "user": { "screen-name": "NilaMilliron_tw", "friends_count": 445, "statuses_count": 164, "name": "Nila Milliron", "followers_count": 22649 }, "sender-location": [37.59, 68.42], "send-time": "2008-03-09T10:10:00.000", "referred-topics": [ "iphone", "platform" ], "message-text": " can't stand iphone its platform is terrible" }
+{ "user": { "screen-name": "OliJackson_512", "friends_count": 445, "statuses_count": 164, "name": "Oli Jackson", "followers_count": 22649 }, "sender-location": [24.82, 94.63], "send-time": "2010-02-13T10:10:00.000", "referred-topics": [ "samsung", "voice-command" ], "message-text": " like samsung the voice-command is amazing:)" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [32.84, 67.14], "send-time": "2010-05-13T10:10:00.000", "referred-topics": [ "verizon", "shortcut-menu" ], "message-text": " like verizon its shortcut-menu is awesome:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [29.72, 75.8], "send-time": "2006-11-04T10:10:00.000", "referred-topics": [ "motorola", "speed" ], "message-text": " like motorola the speed is good:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [39.28, 70.48], "send-time": "2011-12-26T10:10:00.000", "referred-topics": [ "sprint", "voice-command" ], "message-text": " like sprint the voice-command is mind-blowing:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [40.09, 92.69], "send-time": "2006-08-04T10:10:00.000", "referred-topics": [ "motorola", "speed" ], "message-text": " can't stand motorola its speed is terrible:(" }
+{ "user": { "screen-name": "ColineGeyer@63", "friends_count": 121, "statuses_count": 362, "name": "Coline Geyer", "followers_count": 17159 }, "sender-location": [47.51, 83.99], "send-time": "2010-05-07T10:10:00.000", "referred-topics": [ "iphone", "voice-clarity" ], "message-text": " like iphone the voice-clarity is good:)" }
+{ "user": { "screen-name": "ChangEwing_573", "friends_count": 182, "statuses_count": 394, "name": "Chang Ewing", "followers_count": 32136 }, "sender-location": [36.21, 72.6], "send-time": "2011-08-25T10:10:00.000", "referred-topics": [ "samsung", "platform" ], "message-text": " like samsung the platform is good" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [46.05, 93.34], "send-time": "2005-10-14T10:10:00.000", "referred-topics": [ "t-mobile", "shortcut-menu" ], "message-text": " like t-mobile the shortcut-menu is awesome:)" }
+{ "user": { "screen-name": "NathanGiesen@211", "friends_count": 39339, "statuses_count": 473, "name": "Nathan Giesen", "followers_count": 49416 }, "sender-location": [36.86, 74.62], "send-time": "2012-07-21T10:10:00.000", "referred-topics": [ "verizon", "voicemail-service" ], "message-text": " love verizon its voicemail-service is awesome" }
\ No newline at end of file
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
index ea6bac9..7ef8982 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
@@ -19,7 +19,11 @@
package org.apache.asterix.lang.sqlpp.clause;
+import java.util.ArrayList;
+import java.util.List;
import java.util.Objects;
+import java.util.function.Function;
+import java.util.stream.Collectors;
import org.apache.asterix.common.exceptions.CompilationException;
import org.apache.asterix.lang.common.base.AbstractClause;
@@ -27,15 +31,19 @@
import org.apache.asterix.lang.sqlpp.visitor.base.ISqlppVisitor;
public class SelectClause extends AbstractClause {
-
+ private final List<List<String>> fieldExclusions = new ArrayList<>();
private SelectElement selectElement;
private SelectRegular selectRegular;
private boolean distinct;
- public SelectClause(SelectElement selectElement, SelectRegular selectRegular, boolean distinct) {
+ public SelectClause(SelectElement selectElement, SelectRegular selectRegular, List<List<String>> fieldExclusions,
+ boolean distinct) {
this.selectElement = selectElement;
this.selectRegular = selectRegular;
this.distinct = distinct;
+ if (fieldExclusions != null) {
+ this.fieldExclusions.addAll(fieldExclusions);
+ }
}
@Override
@@ -82,15 +90,25 @@
this.distinct = distinct;
}
+ public List<List<String>> getFieldExclusions() {
+ return fieldExclusions;
+ }
+
@Override
public String toString() {
- return "select " + (distinct ? "distinct " : "")
- + (selectElement() ? "element " + selectElement : String.valueOf(selectRegular));
+ String distinctString = distinct ? "distinct " : "";
+ String valueString = selectElement() ? ("element " + selectElement) : String.valueOf(selectRegular);
+ String exceptString = "";
+ if (!fieldExclusions.isEmpty()) {
+ final Function<List<String>, String> fieldBuilder = f -> String.join(".", f);
+ exceptString = " except " + fieldExclusions.stream().map(fieldBuilder).collect(Collectors.joining(", "));
+ }
+ return String.format("select %s%s%s", distinctString, valueString, exceptString);
}
@Override
public int hashCode() {
- return Objects.hash(distinct, selectElement, selectRegular);
+ return Objects.hash(distinct, selectElement, selectRegular, fieldExclusions);
}
@Override
@@ -103,6 +121,7 @@
}
SelectClause target = (SelectClause) object;
return distinct == target.distinct && Objects.equals(selectElement, target.selectElement)
- && Objects.equals(selectRegular, target.selectRegular);
+ && Objects.equals(selectRegular, target.selectRegular)
+ && Objects.equals(fieldExclusions, target.fieldExclusions);
}
}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
index 4352210..71d88fa 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
@@ -130,6 +130,9 @@
// Rewrites RIGHT OUTER JOINs into LEFT OUTER JOINs if possible
rewriteRightJoins();
+
+ // Rewrite SELECT EXCEPT to use OBJECT_REMOVE_FIELDS.
+ rewriteSelectExceptSugar();
}
static Expression castViewBodyAsType(LangRewritingContext context, Expression bodyExpr, IAType itemType,
@@ -218,7 +221,7 @@
RecordConstructor recordConstr = new RecordConstructor(projections);
recordConstr.setSourceLocation(sourceLoc);
- SelectClause selectClause = new SelectClause(new SelectElement(recordConstr), null, false);
+ SelectClause selectClause = new SelectClause(new SelectElement(recordConstr), null, null, false);
selectClause.setSourceLocation(sourceLoc);
SelectBlock selectBlock = new SelectBlock(selectClause, fromClause, letWhereClauseList, null, null);
selectBlock.setSourceLocation(sourceLoc);
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
index 2f4fcc8..93360a9 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
@@ -58,6 +58,7 @@
import org.apache.asterix.lang.sqlpp.rewrites.visitor.InlineColumnAliasVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.InlineWithExpressionVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.OperatorExpressionVisitor;
+import org.apache.asterix.lang.sqlpp.rewrites.visitor.SelectExceptRewriteSugarVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.SetOperationVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.SqlCompatRewriteVisitor;
import org.apache.asterix.lang.sqlpp.rewrites.visitor.SqlppCaseAggregateExtractionVisitor;
@@ -210,6 +211,9 @@
// and external UDFs, therefore need to inline SQL++ UDFs to check the deterministic property.
inlineWithExpressions();
+ // Rewrite SELECT EXCEPT to use OBJECT_REMOVE_FIELDS.
+ rewriteSelectExceptSugar();
+
// Sets the var counter of the query.
topStatement.setVarCounter(context.getVarCounter().get());
}
@@ -350,6 +354,11 @@
}
}
+ protected void rewriteSelectExceptSugar() throws CompilationException {
+ SelectExceptRewriteSugarVisitor selectExceptRewriteSugarVisitor = new SelectExceptRewriteSugarVisitor(context);
+ rewriteTopExpr(selectExceptRewriteSugarVisitor, null);
+ }
+
private <R, T> R rewriteTopExpr(ILangVisitor<R, T> visitor, T arg) throws CompilationException {
R result = topStatement.accept(visitor, arg);
logExpression(">>>> AST After", visitor.getClass().getSimpleName());
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExceptRewriteSugarVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExceptRewriteSugarVisitor.java
new file mode 100644
index 0000000..af8fb6e
--- /dev/null
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExceptRewriteSugarVisitor.java
@@ -0,0 +1,195 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.lang.sqlpp.rewrites.visitor;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Deque;
+import java.util.List;
+import java.util.stream.Collectors;
+
+import org.apache.asterix.common.exceptions.CompilationException;
+import org.apache.asterix.common.functions.FunctionSignature;
+import org.apache.asterix.lang.common.base.Expression;
+import org.apache.asterix.lang.common.base.ILangExpression;
+import org.apache.asterix.lang.common.expression.CallExpr;
+import org.apache.asterix.lang.common.expression.ListConstructor;
+import org.apache.asterix.lang.common.expression.LiteralExpr;
+import org.apache.asterix.lang.common.expression.VariableExpr;
+import org.apache.asterix.lang.common.literal.StringLiteral;
+import org.apache.asterix.lang.common.rewrites.LangRewritingContext;
+import org.apache.asterix.lang.common.struct.VarIdentifier;
+import org.apache.asterix.lang.sqlpp.clause.FromClause;
+import org.apache.asterix.lang.sqlpp.clause.FromTerm;
+import org.apache.asterix.lang.sqlpp.clause.SelectBlock;
+import org.apache.asterix.lang.sqlpp.clause.SelectClause;
+import org.apache.asterix.lang.sqlpp.clause.SelectElement;
+import org.apache.asterix.lang.sqlpp.clause.SelectSetOperation;
+import org.apache.asterix.lang.sqlpp.expression.SelectExpression;
+import org.apache.asterix.lang.sqlpp.struct.SetOperationInput;
+import org.apache.asterix.lang.sqlpp.struct.SetOperationRight;
+import org.apache.asterix.lang.sqlpp.visitor.base.AbstractSqlppExpressionScopingVisitor;
+import org.apache.asterix.om.functions.BuiltinFunctions;
+
+/**
+ * Rewrites the exclusion list of a SELECT clause into OBJECT_REMOVE_FIELDS function applications.
+ * Input:
+ * <pre>
+ * FROM ...
+ * WHERE ...
+ * SELECT * EXCEPT a, b.f
+ * ORDER BY c
+ * </pre>
+ * Output:
+ * <pre>
+ * FROM ( FROM ...
+ * WHERE ...
+ * SELECT *
+ * ORDER BY c ) TMP_1
+ * SELECT VALUE OBJECT_REMOVE_FIELDS(TMP_1, [ "a", [ "b", "f" ]])
+ * </pre>
+ */
+public class SelectExceptRewriteSugarVisitor extends AbstractSqlppExpressionScopingVisitor {
+ private final Deque<SelectExpression> selectExpressionStack = new ArrayDeque<>();
+
+ public SelectExceptRewriteSugarVisitor(LangRewritingContext langRewritingContext) {
+ super(langRewritingContext);
+ }
+
+ /**
+ * We will only return a **new** SELECT-EXPR if we have a SELECT-EXPR with output modifiers, no set operations, and
+ * with a SELECT-BLOCK whose SELECT-CLAUSE contains field exclusions. For all other cases, we will localize our
+ * changes to the affected SELECT-BLOCK.
+ */
+ @Override
+ public Expression visit(SelectExpression selectExpression, ILangExpression arg) throws CompilationException {
+ super.visit(selectExpression, arg);
+ return selectExpressionStack.removeLast();
+ }
+
+ @Override
+ public Expression visit(SelectSetOperation selectSetOperation, ILangExpression arg) throws CompilationException {
+ SelectExpression selectExpression = (SelectExpression) arg;
+ boolean hasOutputModifiers = selectExpression.hasLimit() || selectExpression.hasOrderby();
+ boolean hasRightInputs = selectSetOperation.hasRightInputs();
+ if (hasOutputModifiers && !hasRightInputs) {
+ selectExpressionStack.addLast(wrapSelectExpression(selectExpression));
+ } else {
+ mutateAndWrapSelectBlock(selectSetOperation.getLeftInput(), selectExpression);
+ for (SetOperationRight rightInput : selectSetOperation.getRightInputs()) {
+ mutateAndWrapSelectBlock(rightInput.getSetOperationRightInput(), selectExpression);
+ }
+ selectExpressionStack.addLast(selectExpression);
+ }
+ return null;
+ }
+
+ private SelectExpression wrapSelectExpression(SelectExpression selectExpression) {
+ SelectSetOperation selectSetOperation = selectExpression.getSelectSetOperation();
+ SetOperationInput setOperationInput = selectSetOperation.getLeftInput();
+ if (!setOperationInput.selectBlock()) {
+ return selectExpression;
+ }
+
+ SelectBlock innerSelectBlock = selectExpression.getSelectSetOperation().getLeftInput().getSelectBlock();
+ SelectClause originalSelectClause = innerSelectBlock.getSelectClause();
+ if (originalSelectClause.getFieldExclusions().isEmpty()) {
+ return selectExpression;
+ }
+
+ // Nest our original SELECT-EXPR.
+ SetOperationInput innerSetOpInput = new SetOperationInput(null, selectExpression);
+ SelectSetOperation innerSelectSetOp = new SelectSetOperation(innerSetOpInput, null);
+ SelectExpression innerSelectExpr = new SelectExpression(null, innerSelectSetOp, null, null, true);
+
+ // Build a new SELECT-EXPR.
+ VarIdentifier fromTermVariable = context.newVariable();
+ VariableExpr fromTermVariableExpr = new VariableExpr(fromTermVariable);
+ SelectClause selectClause = buildSelectClause(originalSelectClause, fromTermVariable);
+ FromTerm fromTerm = new FromTerm(innerSelectExpr, fromTermVariableExpr, null, null);
+ FromClause fromClause = new FromClause(List.of(fromTerm));
+ SelectBlock selectBlock = new SelectBlock(selectClause, fromClause, null, null, null);
+ SetOperationInput newSetOpInput = new SetOperationInput(selectBlock, null);
+ SelectSetOperation newSelectSetOperation = new SelectSetOperation(newSetOpInput, null);
+ SelectExpression newSelectExpression = new SelectExpression(null, newSelectSetOperation, null, null, true);
+ newSelectExpression.setSourceLocation(selectExpression.getSourceLocation());
+ return newSelectExpression;
+ }
+
+ private void mutateAndWrapSelectBlock(SetOperationInput setOperationInput, ILangExpression arg)
+ throws CompilationException {
+ Expression visitResult = setOperationInput.accept(this, arg);
+ if (setOperationInput.subquery()) {
+ setOperationInput.setSubquery((SelectExpression) visitResult);
+ }
+
+ // Only proceed if we have fields in our EXCEPT list.
+ SelectBlock innerSelectBlock = setOperationInput.getSelectBlock();
+ SelectClause originalSelectClause = innerSelectBlock.getSelectClause();
+ if (originalSelectClause.getFieldExclusions().isEmpty()) {
+ return;
+ }
+
+ // Nest our original SELECT-BLOCK.
+ SetOperationInput innerSetOpInput = new SetOperationInput(innerSelectBlock, null);
+ SelectSetOperation innerSelectSetOp = new SelectSetOperation(innerSetOpInput, null);
+ SelectExpression innerSelectExpr = new SelectExpression(null, innerSelectSetOp, null, null, true);
+
+ // Build a new SELECT-BLOCK.
+ VarIdentifier fromTermVariable = context.newVariable();
+ VariableExpr fromTermVariableExpr = new VariableExpr(fromTermVariable);
+ SelectClause selectClause = buildSelectClause(originalSelectClause, fromTermVariable);
+ FromTerm fromTerm = new FromTerm(innerSelectExpr, fromTermVariableExpr, null, null);
+ FromClause fromClause = new FromClause(List.of(fromTerm));
+ SelectBlock selectBlock = new SelectBlock(selectClause, fromClause, null, null, null);
+ setOperationInput.setSelectBlock(selectBlock);
+ }
+
+ private SelectClause buildSelectClause(SelectClause originalSelectClause, VarIdentifier iterationVariable) {
+ // Convert our list of identifiers into a list of literals representing field names.
+ ListConstructor listConstructor = new ListConstructor();
+ listConstructor.setType(ListConstructor.Type.ORDERED_LIST_CONSTRUCTOR);
+ listConstructor.setExprList(new ArrayList<>());
+ for (List<String> nestedField : originalSelectClause.getFieldExclusions()) {
+ if (nestedField.size() == 1) {
+ // For non-nested fields, we do not wrap our name in a list.
+ listConstructor.getExprList().add(new LiteralExpr(new StringLiteral(nestedField.get(0))));
+ } else {
+ // Otherwise, build a list to insert into our list.
+ ListConstructor nestedFieldList = new ListConstructor();
+ nestedFieldList.setType(ListConstructor.Type.ORDERED_LIST_CONSTRUCTOR);
+ nestedFieldList.setExprList(nestedField.stream().map(f -> new LiteralExpr(new StringLiteral(f)))
+ .collect(Collectors.toList()));
+ listConstructor.getExprList().add(nestedFieldList);
+ }
+ }
+ List<Expression> objectRemoveFieldsArguments = new ArrayList<>();
+ objectRemoveFieldsArguments.add(new VariableExpr(iterationVariable));
+ objectRemoveFieldsArguments.add(listConstructor);
+
+ // Create the call to OBJECT_REMOVE_FIELDS.
+ FunctionSignature functionSignature = new FunctionSignature(BuiltinFunctions.REMOVE_FIELDS);
+ CallExpr callExpr = new CallExpr(functionSignature, objectRemoveFieldsArguments);
+ SelectElement selectElement = new SelectElement(callExpr);
+ boolean distinct = originalSelectClause.distinct();
+ SelectClause selectClause = new SelectClause(selectElement, null, null, distinct);
+ selectClause.setSourceLocation(originalSelectClause.getSourceLocation());
+ return selectClause;
+ }
+}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java
index 1266d9a..2ff4808 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SetOperationVisitor.java
@@ -112,7 +112,7 @@
newFromClause.setSourceLocation(sourceLoc);
Expression valueExpr = valueExprTransformer != null
? valueExprTransformer.apply(newBindingVar, valueExprTransformerArg) : newBindingVar;
- SelectClause selectClause = new SelectClause(new SelectElement(valueExpr), null, distinct);
+ SelectClause selectClause = new SelectClause(new SelectElement(valueExpr), null, null, distinct);
selectClause.setSourceLocation(sourceLoc);
SelectBlock selectBlock = new SelectBlock(selectClause, newFromClause, null, null, null);
selectBlock.setSourceLocation(sourceLoc);
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java
index f93e843..a2eed4f 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/Sql92AggregateFunctionVisitor.java
@@ -159,7 +159,7 @@
SelectElement selectElement = new SelectElement(newExpr);
selectElement.setSourceLocation(sourceLoc);
- SelectClause selectClause = new SelectClause(selectElement, null, false);
+ SelectClause selectClause = new SelectClause(selectElement, null, null, false);
selectClause.setSourceLocation(sourceLoc);
// Construct the select expression.
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
index b98151b..4ebd50a5 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
@@ -416,7 +416,7 @@
}
RecordConstructor rc1 = new RecordConstructor(fb1List);
rc1.setSourceLocation(sourceLoc);
- SelectClause selectClause1 = new SelectClause(new SelectElement(rc1), null, false);
+ SelectClause selectClause1 = new SelectClause(new SelectElement(rc1), null, null, false);
selectClause1.setSourceLocation(sourceLoc);
SelectBlock newRightSelectBlock = new SelectBlock(selectClause1, fromClause1, null, null, null);
newRightSelectBlock.setSourceLocation(sourceLoc);
@@ -650,7 +650,7 @@
fa.setSourceLocation(sourceLoc);
SelectElement sv1 = new SelectElement(fa);
sv1.setSourceLocation(sourceLoc);
- SelectClause sc1 = new SelectClause(sv1, null, false);
+ SelectClause sc1 = new SelectClause(sv1, null, null, false);
sc1.setSourceLocation(sourceLoc);
SelectBlock sb1 = new SelectBlock(sc1, fc1, null, null, null);
sv1.setSourceLocation(sourceLoc);
@@ -691,7 +691,7 @@
v1Ref2.setSourceLocation(sourceLoc);
SelectElement sv1 = new SelectElement(v1Ref2);
sv1.setSourceLocation(sourceLoc);
- SelectClause sc1 = new SelectClause(sv1, null, false);
+ SelectClause sc1 = new SelectClause(sv1, null, null, false);
sc1.setSourceLocation(sourceLoc);
SelectBlock sb1 = new SelectBlock(sc1, fc1, null, null, null);
sv1.setSourceLocation(sourceLoc);
@@ -743,7 +743,7 @@
SelectElement sv2 = new SelectElement(iaExpr);
sv2.setSourceLocation(sourceLoc);
- SelectClause sc2 = new SelectClause(sv2, null, false);
+ SelectClause sc2 = new SelectClause(sv2, null, null, false);
sc2.setSourceLocation(sourceLoc);
List<AbstractClause> sb2Clauses = new ArrayList<>(1);
sb2Clauses.add(lc2);
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java
index 1259b64..5a565be 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppGroupingSetsVisitor.java
@@ -411,7 +411,8 @@
.add(new Projection(Projection.Kind.NAMED_EXPR, selectElement.getExpression(), mainProjectionName));
SelectRegular newSelectRegular = new SelectRegular(projectionList);
newSelectRegular.setSourceLocation(selectElement.getSourceLocation());
- SelectClause newSelectClause = new SelectClause(null, newSelectRegular, selectClause.distinct());
+ SelectClause newSelectClause =
+ new SelectClause(null, newSelectRegular, selectClause.getFieldExclusions(), selectClause.distinct());
newSelectClause.setSourceLocation(selectClause.getSourceLocation());
return newSelectClause;
}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java
index 26c6915..e6042f4 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppRightJoinRewriteVisitor.java
@@ -354,7 +354,7 @@
newFromTermList.add(newFromTerm);
FromClause newFromClause = new FromClause(newFromTermList);
newFromClause.setSourceLocation(sourceLoc);
- SelectClause newSelectClause = new SelectClause(null, new SelectRegular(projectList), false);
+ SelectClause newSelectClause = new SelectClause(null, new SelectRegular(projectList), null, false);
newSelectClause.setSourceLocation(sourceLoc);
SelectBlock newSelectBlock = new SelectBlock(newSelectClause, newFromClause, null, null, null);
newSelectBlock.setSourceLocation(sourceLoc);
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
index d74d047..60d93b4 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
@@ -193,7 +193,8 @@
if (selectClause.selectRegular()) {
selectRegular = (SelectRegular) selectClause.getSelectRegular().accept(this, arg);
}
- SelectClause copy = new SelectClause(selectElement, selectRegular, selectClause.distinct());
+ List<List<String>> fieldExclusions = selectClause.getFieldExclusions();
+ SelectClause copy = new SelectClause(selectElement, selectRegular, fieldExclusions, selectClause.distinct());
copy.setSourceLocation(selectClause.getSourceLocation());
return copy;
}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
index ce0d0a1..63c89f6 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
@@ -263,13 +263,15 @@
if (selectClause.selectElement()) {
Pair<ILangExpression, VariableSubstitutionEnvironment> newSelectElement =
selectClause.getSelectElement().accept(this, env);
- SelectClause newSelectClause = new SelectClause((SelectElement) newSelectElement.first, null, distinct);
+ SelectClause newSelectClause = new SelectClause((SelectElement) newSelectElement.first, null,
+ selectClause.getFieldExclusions(), distinct);
newSelectClause.setSourceLocation(selectClause.getSourceLocation());
return new Pair<>(newSelectClause, newSelectElement.second);
} else {
Pair<ILangExpression, VariableSubstitutionEnvironment> newSelectRegular =
selectClause.getSelectRegular().accept(this, env);
- SelectClause newSelectClause = new SelectClause(null, (SelectRegular) newSelectRegular.first, distinct);
+ SelectClause newSelectClause = new SelectClause(null, (SelectRegular) newSelectRegular.first,
+ selectClause.getFieldExclusions(), distinct);
newSelectClause.setSourceLocation(selectClause.getSourceLocation());
return new Pair<>(newSelectClause, newSelectRegular.second);
}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java
index 7e7b654..c313f06 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppDeleteRewriteVisitor.java
@@ -89,7 +89,7 @@
returnExpr.setSourceLocation(var.getSourceLocation());
SelectElement selectElement = new SelectElement(returnExpr);
selectElement.setSourceLocation(deleteStmt.getSourceLocation());
- SelectClause selectClause = new SelectClause(selectElement, null, false);
+ SelectClause selectClause = new SelectClause(selectElement, null, null, false);
selectClause.setSourceLocation(deleteStmt.getSourceLocation());
// Construct the select expression.
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
index a37b9cc..52646c7 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
+++ b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
@@ -4724,6 +4724,9 @@
Token startToken = null;
SelectRegular selectRegular = null;
SelectElement selectElement = null;
+ List<List<String>> fieldExclusions = new ArrayList<List<String>>();
+ List<String> nestedField = new ArrayList<String>();
+ String identifier;
boolean distinct = false;
}
{
@@ -4733,6 +4736,23 @@
|
selectElement = SelectElement()
)?
+ (
+ <EXCEPT>
+ identifier = Identifier() { nestedField.add(identifier); }
+ ( <DOT> identifier = Identifier() { nestedField.add(identifier); } )*
+ {
+ fieldExclusions.add(nestedField);
+ nestedField = new ArrayList<String>();
+ }
+ (
+ <COMMA> identifier = Identifier() { nestedField.add(identifier); }
+ ( <DOT> identifier = Identifier() { nestedField.add(identifier); } )*
+ {
+ fieldExclusions.add(nestedField);
+ nestedField = new ArrayList<String>();
+ }
+ )*
+ )?
{
SourceLocation sourceLoc = getSourceLocation(startToken);
if (selectRegular == null && selectElement == null){
@@ -4743,7 +4763,7 @@
selectRegular = new SelectRegular(projections);
selectRegular.setSourceLocation(sourceLoc);
}
- SelectClause selectClause = new SelectClause(selectElement, selectRegular, distinct);
+ SelectClause selectClause = new SelectClause(selectElement, selectRegular, fieldExclusions, distinct);
selectClause.setSourceLocation(sourceLoc);
return selectClause;
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-MessageType: newchange
Change in asterixdb[master]: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:
Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
......................................................................
Patch Set 1:
Graphix Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/graphix-compat/210/
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-CC: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 11 Aug 2022 00:08:41 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment
Change in asterixdb[master]: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:
Attention is currently required from: Glenn Galvizo.
Hello Jenkins, Anon. E. Moose #1000171,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
to look at the new patch set (#5).
Change subject: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
......................................................................
[WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
- user model changes: yes
- storage format changes: no
- interface changes: no
details:
- adding syntactic sugar rewrite to specify what fields to
EXCLUDE in a SELECT clause
- fixing result type computer for OBJECT_REMOVE_FIELDS.
Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
---
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordRemoveFieldsTypeComputer.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-exclude/select-exclude.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-exclude/select-exclude.5.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-exclude/select-exclude.3.adm
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-exclude/select-exclude.3.query.sqlpp
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-exclude/select-exclude.6.adm
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-exclude/select-exclude.4.query.sqlpp
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-exclude/select-exclude.5.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-exclude/select-exclude.6.query.sqlpp
M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-exclude/select-exclude.4.adm
M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
A asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExcludeRewriteSugarVisitor.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-exclude/select-exclude.1.ddl.sqlpp
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
20 files changed, 586 insertions(+), 11 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/17103/5
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 5
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Attention: Glenn Galvizo <gg...@uci.edu>
Gerrit-MessageType: newpatchset
Change in asterixdb[master]: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Attention is currently required from: Glenn Galvizo.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
......................................................................
Patch Set 5:
Analytics Compatibility Compilation Successful
: SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 5
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Attention: Glenn Galvizo <gg...@uci.edu>
Gerrit-Comment-Date: Sat, 01 Oct 2022 17:08:28 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment
Change in asterixdb[master]: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Attention is currently required from: Glenn Galvizo.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
......................................................................
Patch Set 5: Contrib+1
Analytics Compatibility Tests Successful
: SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 5
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Attention: Glenn Galvizo <gg...@uci.edu>
Gerrit-Comment-Date: Sat, 01 Oct 2022 19:08:27 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:
Attention is currently required from: Glenn Galvizo.
Hello Jenkins, Anon. E. Moose #1000171,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
to look at the new patch set (#9).
Change subject: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
......................................................................
[ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
- user model changes: yes
- storage format changes: no
- interface changes: no
details:
- adding syntactic sugar rewrite to specify what fields to
EXCLUDE in a SELECT clause
- fixing result type computer for OBJECT_REMOVE_FIELDS.
Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
---
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-star/select-exclude/select-exclude.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-star/select-exclude/select-exclude.4.adm
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordRemoveFieldsTypeComputer.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-star/select-exclude/select-exclude.6.adm
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-star/select-exclude/select-exclude.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-star/select-exclude/select-exclude.5.query.sqlpp
M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/sqlpp/ParserTestExecutor.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-star/select-exclude/select-exclude.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-star/select-exclude/select-exclude.1.ddl.sqlpp
M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
A asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SelectExcludeRewriteSugarVisitor.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/select-star/select-exclude/select-exclude.6.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-star/select-exclude/select-exclude.3.adm
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppFunctionBodyRewriter.java
A asterixdb/asterix-app/src/test/resources/runtimets/results/select-star/select-exclude/select-exclude.5.adm
20 files changed, 560 insertions(+), 11 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/17103/9
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 9
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Attention: Glenn Galvizo <gg...@uci.edu>
Gerrit-MessageType: newpatchset
Change in asterixdb[master]: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Attention is currently required from: Glenn Galvizo.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [WIP][ASTERIXDB-3059][SQL] Support for SELECT .. EXCEPT ..
......................................................................
Patch Set 2: Contrib-2
Analytics Compatibility Tests Failed
: UNSTABLE
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 2
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Attention: Glenn Galvizo <gg...@uci.edu>
Gerrit-Comment-Date: Wed, 28 Sep 2022 06:50:56 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:
Attention is currently required from: Glenn Galvizo, Till Westmann.
Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
......................................................................
Patch Set 14: Integration-Tests+1
Integration Tests Successful
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests-redux/16/ : null
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 14
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@couchbase.com>
Gerrit-Attention: Glenn Galvizo <gg...@uci.edu>
Gerrit-Attention: Till Westmann <ti...@couchbase.com>
Gerrit-Comment-Date: Sun, 16 Oct 2022 23:39:53 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:
Attention is currently required from: Ali Alsuliman, Till Westmann.
Glenn Galvizo has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
......................................................................
Patch Set 15:
(6 comments)
Patchset:
PS10:
> Can we add these test cases (whether negative or positive)? […]
all test cases have been added, and a few bugs have been caught and fixed:
1. update to the grammar to not recognize EXCLUDE as a an alias for a projection
2. single-variable-rule fixes (we recognize variables outside of local scope)
3. unknown handling fixes in OBJECT_REMOVE_FIELDS type computer and runtime -- required to handle LOJ and ROJ
File asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/SelectClause.java:
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103/comment/2c0646b0_d8dcda78
PS10, Line 34: private final List<List<String>> fieldExclusions = new ArrayList<>();
> now that we are adding a new piece of information, we should check all the visitors of SelectClause […]
Ack
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103/comment/7ba8650d_3b70c5df
PS10, Line 41: this.selectElement = selectElement;
: this.selectRegular = selectRegular;
> not related to this change, but I believe these two cannot be specified together. […]
Ack
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103/comment/c5e65321_500690d5
PS10, Line 44: if (fieldExclusions != null) {
> it would be good to also check that if fieldExclusions is not null, then selectElement must be null […]
Ack
File asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/DeepCopyVisitor.java:
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103/comment/1a2c99be_9f2f4293
PS10, Line 197: List<List<String>> fieldExclusions = selectClause.getFieldExclusions();
> to make it a true deep copy, we should also copy the elements inside. […]
Ack
File asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppCloneAndSubstituteVariablesVisitor.java:
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103/comment/be2e3dd3_18d96fa4
PS10, Line 273: selectClause.getFieldExclusions()
> Same comment as in DeepCopyVisitor.
Ack
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 15
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@couchbase.com>
Gerrit-Attention: Ali Alsuliman <al...@gmail.com>
Gerrit-Attention: Till Westmann <ti...@couchbase.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 21:15:23 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: Ali Alsuliman <al...@gmail.com>
Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Attention is currently required from: Glenn Galvizo, Till Westmann.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103 )
Change subject: [ASTERIXDB-3059][SQL] Support for SELECT .. EXCLUDE ..
......................................................................
Patch Set 12: Contrib-2
Analytics Compatibility Tests Failed
: UNSTABLE
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17103
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ib076d130d89077964396fcbf51602488c9e90682
Gerrit-Change-Number: 17103
Gerrit-PatchSet: 12
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@couchbase.com>
Gerrit-Attention: Glenn Galvizo <gg...@uci.edu>
Gerrit-Attention: Till Westmann <ti...@couchbase.com>
Gerrit-Comment-Date: Fri, 14 Oct 2022 00:22:09 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment