You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by "tmater (via GitHub)" <gi...@apache.org> on 2026/03/24 16:15:03 UTC
[PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
tmater opened a new pull request, #4844:
URL: https://github.com/apache/calcite/pull/4844
## Changes Proposed
This PR adds opt-in support for `:` as a field/item access operator behind a new `SqlConformance#isColonFieldAccessAllowed()` hook. No built-in conformance enables it yet, so the default parser behavior is unchanged.
It also refactors postfix parsing so colon field access composes with existing dot and bracket access. That covers cases such as `v:field`, `v:['field name']`, `arr[1]:field`, `obj['x']:nested['y']`, and colon access followed by bracket or dot postfixes.
To avoid ambiguity when colon field access is enabled, `JSON_OBJECT` and `JSON_OBJECTAGG` must use `VALUE` syntax instead of `:` or comma-separated key/value shorthand.
The Babel parser coverage is updated as well so PostgreSQL-style infix casts still require parentheses before bracket access. For example, `(v::variant)[1]` parses as item access on the cast result, while `v::variant[1]` remains invalid.
## Reproduction
```sql
select v:field, v:['field name'], arr[1]:field, obj['x']:nested from t;
select json_object(key v:field value arr[1]:field);
select (v::variant)[1] from t;
```
## Testing
Added parser coverage in `SqlParserTest` and `BabelParserTest` for valid and invalid colon field access, JSON constructor disambiguation, and `::` cast interactions.
## Jira Link
[CALCITE-7448](https://issues.apache.org/jira/browse/CALCITE-7448)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3045060922
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -3791,13 +3809,61 @@ void AddExpression2b(List<Object> list, ExprContext exprContext) :
e = Expression3(exprContext) {
list.add(e);
}
+ AddRegularPostfixes(list)
+ [
+ LOOKAHEAD(2, <COLON> SimpleIdentifier(),
+ { this.conformance.isColonFieldAccessAllowed() })
+ <COLON>
Review Comment:
I kept `:` intentionally non-chainable here. After one `:` we are already back in the existing postfix world, so the follow-up access can be expressed with the normal operators:
- `a:b.c` instead of `a:b:c`
- `a:['x'].y` instead of `a:['x']:y`
- `(a:b)['x']` / `a:b['x']` instead of `a:b:['x']`
So a second `:` would mostly be extra surface syntax, not extra expressiveness. Keeping it to a single colon also keeps the grammar narrower and avoids widening the ambiguous space around other colon uses, especially `::` in Babel and JSON constructor `:` handling.
If we decide later that repeated `:` is a real dialect requirement, we can extend the current `[...]` to a loop, but I wanted to start with the minimal syntax that covers the targeted cases.
##########
babel/src/test/java/org/apache/calcite/test/BabelParserTest.java:
##########
@@ -301,6 +306,100 @@ private void checkParseInfixCast(String sqlType) {
sql(sql).ok(expected);
}
+ @Test void testColonFieldAccessWithInfixCast() {
+ final SqlParserFixture f =
+ fixture().withConformance(new SqlAbstractConformance() {
+ @Override public boolean isColonFieldAccessAllowed() {
+ return true;
+ }
+ });
+
+ // Bracket after :: binds to the type, not as subscript on the cast result
+ sql("select v::varchar[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+ f.sql("select v::varchar[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+
+ // Explicit parentheses make no difference — bracket still binds to the type
+ sql("select (v::varchar)[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+ f.sql("select (v::varchar)[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+
+ // Array type with bracket
+ sql("select v::integer array[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+ f.sql("select v::integer array[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+
+ // Parenthesized array type with bracket
+ sql("select (v::integer array)[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+ f.sql("select (v::integer array)[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+
+ // Multiple brackets bind to the type
+ sql("select v::varchar[1][2] from t")
+ .ok("SELECT `V` :: VARCHAR[1][2]\nFROM `T`");
+ f.sql("select v::varchar[1][2] from t")
+ .ok("SELECT `V` :: VARCHAR[1][2]\nFROM `T`");
+
+ // Expression on the LHS of ::
+ sql("select (v + 1)::integer[1] from t")
+ .ok("SELECT (`V` + 1) :: INTEGER[1]\nFROM `T`");
+ f.sql("select (v + 1)::integer[1] from t")
+ .ok("SELECT (`V` + 1) :: INTEGER[1]\nFROM `T`");
+
+ // Plain dot: [n].field correctly means subscript-then-field
+ sql("select v.field[1].field2 from t")
+ .ok("SELECT (`V`.`FIELD`[1].`FIELD2`)\nFROM `T`");
+ f.sql("select v:field[1].field2 from t")
+ .ok("SELECT ((`V`.`FIELD`)[1].`FIELD2`)\nFROM `T`");
+
+ // With ::, [n].field is consumed as part of the type
+ sql("select v::varchar[1].field from t")
+ .ok("SELECT `V` :: (VARCHAR[1].`FIELD`)\nFROM `T`");
+ f.sql("select v:field::varchar[1].field2 from t")
+ .ok("SELECT (`V`.`FIELD`) :: (VARCHAR[1].`FIELD2`)\nFROM `T`");
+
+ // Explicit parentheses around :: let [n].field apply to the cast result
+ sql("select (v::varchar)[1].field from t")
+ .ok("SELECT (`V` :: VARCHAR[1].`FIELD`)\nFROM `T`");
+ f.sql("select (v:field::varchar)[1].field2 from t")
+ .ok("SELECT ((`V`.`FIELD`) :: VARCHAR[1].`FIELD2`)\nFROM `T`");
+
+ // Bracket on the result of :: cast
+ sql("select v::integer[1] from t")
+ .ok("SELECT `V` :: INTEGER[1]\nFROM `T`");
+ f.sql("select v:field::integer[1] from t")
+ .ok("SELECT (`V`.`FIELD`) :: INTEGER[1]\nFROM `T`");
+
+ // Dot/bracket access combined with infix cast
+ sql("select v.field::integer,\n"
+ + " arr[1].field::varchar,\n"
+ + " v.field.field2::integer,\n"
+ + " v.field[2]::integer\n"
+ + "from t")
+ .ok("SELECT `V`.`FIELD` :: INTEGER,"
+ + " (`ARR`[1].`FIELD`) :: VARCHAR,"
+ + " `V`.`FIELD`.`FIELD2` :: INTEGER,"
+ + " `V`.`FIELD`[2] :: INTEGER\n"
+ + "FROM `T`");
+ f.sql("select v:field::integer,\n"
+ + " arr[1]:field::varchar,\n"
+ + " v:field.field2::integer,\n"
+ + " v:field[2]::integer\n"
+ + "from t")
+ .ok("SELECT (`V`.`FIELD`) :: INTEGER,"
+ + " (`ARR`[1].`FIELD`) :: VARCHAR,"
+ + " ((`V`.`FIELD`).`FIELD2`) :: INTEGER,"
+ + " (`V`.`FIELD`)[2] :: INTEGER\n"
+ + "FROM `T`");
+
+ // Double-colon followed by colon field access is not allowed
+ f.sql("select v::variant^:^field from t")
+ .fails("(?s).*Encountered \":.*\".*");
+ }
Review Comment:
Done
##########
babel/src/main/codegen/includes/parserImpls.ftl:
##########
@@ -208,6 +208,7 @@ void InfixCast(List<Object> list, ExprContext exprContext, Span s) :
s.pos()));
list.add(dt);
}
+ AddRegularPostfixes(list)
Review Comment:
Before the refactor, `InfixCast` could stop after `DataType()` and rely on the outer `Expression2()` loop to notice any following `.` or `[...]`. After the refactor, that outer postfix branch was removed and the shared postfix handling moved earlier into `AddExpression2b()`. But Babel `InfixCast` is not part of that earlier path; it is an `extraBinaryExpressions` hook that runs later.
So if `InfixCast` does not invoke `AddRegularPostfixes()` itself, nobody else will. That is why it has to “own” it now: not because the semantics changed, but because the control flow changed. The shared postfix parser still exists, but this is now the only place on the Babel `::` path where it can be called.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4244023547
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "caicancai (via GitHub)" <gi...@apache.org>.
caicancai commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3036509827
##########
babel/src/test/java/org/apache/calcite/test/BabelParserTest.java:
##########
@@ -301,6 +306,100 @@ private void checkParseInfixCast(String sqlType) {
sql(sql).ok(expected);
}
+ @Test void testColonFieldAccessWithInfixCast() {
+ final SqlParserFixture f =
+ fixture().withConformance(new SqlAbstractConformance() {
+ @Override public boolean isColonFieldAccessAllowed() {
+ return true;
+ }
+ });
+
+ // Bracket after :: binds to the type, not as subscript on the cast result
+ sql("select v::varchar[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+ f.sql("select v::varchar[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+
+ // Explicit parentheses make no difference — bracket still binds to the type
+ sql("select (v::varchar)[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+ f.sql("select (v::varchar)[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+
+ // Array type with bracket
+ sql("select v::integer array[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+ f.sql("select v::integer array[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+
+ // Parenthesized array type with bracket
+ sql("select (v::integer array)[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+ f.sql("select (v::integer array)[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+
+ // Multiple brackets bind to the type
+ sql("select v::varchar[1][2] from t")
+ .ok("SELECT `V` :: VARCHAR[1][2]\nFROM `T`");
+ f.sql("select v::varchar[1][2] from t")
+ .ok("SELECT `V` :: VARCHAR[1][2]\nFROM `T`");
+
+ // Expression on the LHS of ::
+ sql("select (v + 1)::integer[1] from t")
+ .ok("SELECT (`V` + 1) :: INTEGER[1]\nFROM `T`");
+ f.sql("select (v + 1)::integer[1] from t")
+ .ok("SELECT (`V` + 1) :: INTEGER[1]\nFROM `T`");
+
+ // Plain dot: [n].field correctly means subscript-then-field
+ sql("select v.field[1].field2 from t")
+ .ok("SELECT (`V`.`FIELD`[1].`FIELD2`)\nFROM `T`");
+ f.sql("select v:field[1].field2 from t")
+ .ok("SELECT ((`V`.`FIELD`)[1].`FIELD2`)\nFROM `T`");
+
+ // With ::, [n].field is consumed as part of the type
+ sql("select v::varchar[1].field from t")
+ .ok("SELECT `V` :: (VARCHAR[1].`FIELD`)\nFROM `T`");
+ f.sql("select v:field::varchar[1].field2 from t")
+ .ok("SELECT (`V`.`FIELD`) :: (VARCHAR[1].`FIELD2`)\nFROM `T`");
+
+ // Explicit parentheses around :: let [n].field apply to the cast result
+ sql("select (v::varchar)[1].field from t")
+ .ok("SELECT (`V` :: VARCHAR[1].`FIELD`)\nFROM `T`");
+ f.sql("select (v:field::varchar)[1].field2 from t")
+ .ok("SELECT ((`V`.`FIELD`) :: VARCHAR[1].`FIELD2`)\nFROM `T`");
+
+ // Bracket on the result of :: cast
+ sql("select v::integer[1] from t")
+ .ok("SELECT `V` :: INTEGER[1]\nFROM `T`");
+ f.sql("select v:field::integer[1] from t")
+ .ok("SELECT (`V`.`FIELD`) :: INTEGER[1]\nFROM `T`");
+
+ // Dot/bracket access combined with infix cast
+ sql("select v.field::integer,\n"
+ + " arr[1].field::varchar,\n"
+ + " v.field.field2::integer,\n"
+ + " v.field[2]::integer\n"
+ + "from t")
+ .ok("SELECT `V`.`FIELD` :: INTEGER,"
+ + " (`ARR`[1].`FIELD`) :: VARCHAR,"
+ + " `V`.`FIELD`.`FIELD2` :: INTEGER,"
+ + " `V`.`FIELD`[2] :: INTEGER\n"
+ + "FROM `T`");
+ f.sql("select v:field::integer,\n"
+ + " arr[1]:field::varchar,\n"
+ + " v:field.field2::integer,\n"
+ + " v:field[2]::integer\n"
+ + "from t")
+ .ok("SELECT (`V`.`FIELD`) :: INTEGER,"
+ + " (`ARR`[1].`FIELD`) :: VARCHAR,"
+ + " ((`V`.`FIELD`).`FIELD2`) :: INTEGER,"
+ + " (`V`.`FIELD`)[2] :: INTEGER\n"
+ + "FROM `T`");
+
+ // Double-colon followed by colon field access is not allowed
+ f.sql("select v::variant^:^field from t")
+ .fails("(?s).*Encountered \":.*\".*");
+ }
Review Comment:
You can add a blank line.
##########
babel/src/main/codegen/includes/parserImpls.ftl:
##########
@@ -208,6 +208,7 @@ void InfixCast(List<Object> list, ExprContext exprContext, Span s) :
s.pos()));
list.add(dt);
}
+ AddRegularPostfixes(list)
Review Comment:
I'm not sure if it will affect existing semantics.
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -3791,13 +3809,61 @@ void AddExpression2b(List<Object> list, ExprContext exprContext) :
e = Expression3(exprContext) {
list.add(e);
}
+ AddRegularPostfixes(list)
+ [
+ LOOKAHEAD(2, <COLON> SimpleIdentifier(),
+ { this.conformance.isColonFieldAccessAllowed() })
+ <COLON>
Review Comment:
Why is chained colon access not supported?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3100545931
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -3774,6 +3774,24 @@ SqlNode Expression(ExprContext exprContext) :
list = Expression2(exprContext) { return SqlParserUtil.toTree(list); }
}
+void AddRegularPostfixes(List<Object> list) :
+{
+ SqlNode ext;
+}
+{
+ (
+ LOOKAHEAD(2) <DOT>
+ ext = RowExpressionExtension() {
+ list.add(
+ new SqlParserUtil.ToTreeListItem(
+ SqlStdOperatorTable.DOT, getPos()));
Review Comment:
I ran into this downstream while building on top of this work. It gets hard later to tell whether the original syntax was `:` or `.`, since : is rewritten to DOT / ITEM immediately, so I think a dedicated follow-up makes sense there. I’d still prefer to keep this PR scoped to the parser change, and I opened CALCITE-7476 for the next step.
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -3774,6 +3774,48 @@ SqlNode Expression(ExprContext exprContext) :
list = Expression2(exprContext) { return SqlParserUtil.toTree(list); }
}
+/** Parses regular postfixes such as {@code a.b}, {@code a[1]} and
+ * {@code a[1].b}. Shared by the main expression grammar and Babel's
+ * infix cast. */
+void AddRegularPostfixes(List<Object> list) :
+{
+ SqlNode ext;
+}
+{
+ (
+ LOOKAHEAD(2) <DOT>
+ ext = RowExpressionExtension() {
+ list.add(
+ new SqlParserUtil.ToTreeListItem(
+ SqlStdOperatorTable.DOT, getPos()));
+ list.add(ext);
+ }
+ |
+ AddBracketPostfix(list)
+ )*
+}
+
+/** Parses postfixes that are allowed after colon field access, for example
+ * {@code v:field.nested} or {@code v:[1]['x']}. Unlike regular postfixes,
+ * this path does not allow member-style calls such as {@code v:field.func()}. */
+void AddColonPostfixes(List<Object> list) :
Review Comment:
I added a separate postfix method for colon access because this came up during downstream review. Reusing the regular postfix path was too permissive, since it would also allow the same member-style calls as dot access, while for `:` we only want path continuation with `.` and `[]`; Spark/Databricks blocks that broader dot-style behavior as well.
##########
babel/src/test/java/org/apache/calcite/test/BabelParserTest.java:
##########
@@ -301,6 +306,101 @@ private void checkParseInfixCast(String sqlType) {
sql(sql).ok(expected);
}
+ @Test void testColonFieldAccessWithInfixCast() {
+ final SqlParserFixture f =
+ fixture().withConformance(new SqlAbstractConformance() {
+ @Override public boolean isColonFieldAccessAllowed() {
+ return true;
+ }
+ });
+
+ // Bracket after :: binds to the type, not as subscript on the cast result
+ sql("select v::varchar array[1] from t")
+ .ok("SELECT `V` :: VARCHAR ARRAY[1]\nFROM `T`");
+ f.sql("select v::varchar array[1] from t")
Review Comment:
Filed [CALCITE-7475](https://issues.apache.org/jira/browse/CALCITE-7475) to cover this part and narrowed down the test surface.
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -3791,13 +3809,61 @@ void AddExpression2b(List<Object> list, ExprContext exprContext) :
e = Expression3(exprContext) {
list.add(e);
}
+ AddRegularPostfixes(list)
+ [
+ LOOKAHEAD(2, <COLON> SimpleIdentifier(),
+ { this.conformance.isColonFieldAccessAllowed() })
+ <COLON>
+ ext = SimpleIdentifier() {
+ list.add(
+ new SqlParserUtil.ToTreeListItem(
+ SqlStdOperatorTable.DOT, getPos()));
+ list.add(ext);
+ }
+ AddRegularPostfixes(list)
+ |
+ LOOKAHEAD(2, <COLON> <LBRACKET>,
+ { this.conformance.isColonFieldAccessAllowed() })
+ <COLON>
+ AddBracketAccess(list)
+ AddRegularPostfixes(list)
+ ]
+}
+
+void AddBracketAccess(List<Object> list) :
Review Comment:
Done
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -3791,13 +3809,61 @@ void AddExpression2b(List<Object> list, ExprContext exprContext) :
e = Expression3(exprContext) {
list.add(e);
}
+ AddRegularPostfixes(list)
+ [
+ LOOKAHEAD(2, <COLON> SimpleIdentifier(),
+ { this.conformance.isColonFieldAccessAllowed() })
+ <COLON>
Review Comment:
Yes, for the subset this patch is targeting, it covers the common Snowflake/Databricks colon-path surface.
Snowflake is proprietary, I could only reverse engineer/check docs. Examples in the docs are `src:salesperson.name`, `src:vehicle[0].price`, and `src:vehicle[0].price::NUMBER`, which matches the syntax this change adds. But experimented with it and `s:payload.score::INT` works too.
For Databricks, the public docs and the open Spark grammar match this implementation well. Spark treats `:` as semi-structured field access and `::` as a separate cast operator. That matches the forms added here, such as `v:field`, `v:['field']`, `arr[1]:field`, `v:field[1]`, and `v:field::int`.
Reference: [SqlBaseParser.g4#L1316](https://github.com/apache/spark/blob/master/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4#L1316)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3166644077
##########
core/src/main/java/org/apache/calcite/sql/fun/SqlColonOperator.java:
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlSpecialOperator;
+import org.apache.calcite.sql.SqlWriter;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.sql.util.SqlBasicVisitor;
+import org.apache.calcite.sql.util.SqlVisitor;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+import org.apache.calcite.util.Litmus;
+
+import java.util.Arrays;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * The colon operator {@code :}, which preserves field/item path access syntax
+ * in the SQL tree.
+ *
+ * <p>Operands are {@code (base, path)}, where {@code path} is a
+ * {@link SqlNodeList} of structural segments only: identifiers (dot-notation),
+ * character literals (bracketed string keys), or numeric literals (bracketed
+ * indexes).
+ *
+ * <p>Calcite does not lower this operator to Rex. Engines must register a
Review Comment:
Agreed, dropped the convertlet/engine paragraphs. The class Javadoc now describes only the operator and its operand shape:
##########
core/src/main/java/org/apache/calcite/sql/fun/SqlColonOperator.java:
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlSpecialOperator;
+import org.apache.calcite.sql.SqlWriter;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.sql.util.SqlBasicVisitor;
+import org.apache.calcite.sql.util.SqlVisitor;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+import org.apache.calcite.util.Litmus;
+
+import java.util.Arrays;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * The colon operator {@code :}, which preserves field/item path access syntax
+ * in the SQL tree.
+ *
+ * <p>Operands are {@code (base, path)}, where {@code path} is a
+ * {@link SqlNodeList} of structural segments only: identifiers (dot-notation),
+ * character literals (bracketed string keys), or numeric literals (bracketed
+ * indexes).
+ *
+ * <p>Calcite does not lower this operator to Rex. Engines must register a
+ * convertlet for {@link SqlKind#COLON}; without one, sql2rel fails.
+ */
+public class SqlColonOperator extends SqlSpecialOperator {
+ SqlColonOperator() {
+ super("COLON", SqlKind.COLON, 100, true, null, null, null);
+ }
+
+ // Path segments are structural literals/identifiers, never column refs, so
+ // they must not be routed through expression visitors such as AggChecker.
+ @Override public <R> void acceptCall(SqlVisitor<R> visitor, SqlCall call,
+ boolean onlyExpressions, SqlBasicVisitor.ArgHandler<R> argHandler) {
+ if (onlyExpressions) {
+ argHandler.visitChild(visitor, call, 0, call.operand(0));
+ } else {
+ super.acceptCall(visitor, call, onlyExpressions, argHandler);
+ }
+ }
+
+ @Override public ReduceResult reduceExpr(int ordinal, TokenSequence list) {
+ final SqlNode left = list.node(ordinal - 1);
+ final SqlNode right = list.node(ordinal + 1);
+ return new ReduceResult(ordinal - 1, ordinal + 2,
+ createCall(
+ SqlParserPos.sum(
+ Arrays.asList(
+ requireNonNull(left, "left").getParserPosition(),
+ requireNonNull(right, "right").getParserPosition(),
+ list.pos(ordinal))),
+ left,
+ right));
+ }
+
+ @Override public void unparse(SqlWriter writer, SqlCall call, int leftPrec,
+ int rightPrec) {
+ final SqlWriter.Frame frame =
+ writer.startList(SqlWriter.FrameTypeEnum.IDENTIFIER);
+ call.operand(0).unparse(writer, leftPrec, 0);
+ writer.setNeedWhitespace(false);
+ writer.keyword(":");
+ writer.setNeedWhitespace(false);
+ boolean first = true;
+ for (SqlNode segment : (SqlNodeList) call.operand(1)) {
+ if (segment instanceof SqlIdentifier) {
+ if (!first) {
+ writer.sep(".");
+ }
+ segment.unparse(writer, 0, 0);
+ } else {
+ final SqlWriter.Frame b = writer.startList("[", "]");
+ segment.unparse(writer, 0, 0);
+ writer.endList(b);
+ }
+ first = false;
+ }
+ writer.endList(frame);
+ }
+
+ @Override public SqlOperandCountRange getOperandCountRange() {
+ return SqlOperandCountRanges.of(2);
+ }
+
+ // Returns nullable ANY: path access can yield NULL on a missing key, and
+ // the concrete type is up to the engine's convertlet.
Review Comment:
Good call. Looking at how this is used in practice (Databricks/Snowflake), the result of a colon path is always semi-structured, Databricks specifically types it as VARIANT. For our purposes that's enough to commit to "result is always nullable VARIANT" and reject non-VARIANT bases up front. Updated deriveType accordingly
The same VARIANT-in/VARIANT-out shape already exists in [SqlItemOperator.inferReturnType](https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java#L215) and [SqlDotOperator.deriveType](https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/sql/fun/SqlDotOperator.java#L105), so we're consistent with how Calcite handles variant elsewhere.
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlConformance.java:
##########
@@ -260,6 +260,21 @@ enum SelectAliasLookup {
*/
boolean allowHyphenInUnquotedTableName();
+ /**
+ * Whether {@code :} is allowed as a field/item access operator.
+ *
+ * <p>If true, expressions such as {@code v:field} and {@code v:['field']}
+ * are valid. In this mode, JSON constructors must use the
+ * {@code KEY ... VALUE} form rather than {@code :} or comma-pair syntax.
+ *
+ * <p>No built-in conformance level enables this; engines that want
Review Comment:
Done, dropped the references to convertlets and engine-side decisions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4351194927
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [4 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [96.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3181909376
##########
core/src/main/java/org/apache/calcite/sql/fun/SqlColonOperator.java:
##########
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlSpecialOperator;
+import org.apache.calcite.sql.SqlUtil;
+import org.apache.calcite.sql.SqlWriter;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.sql.util.SqlBasicVisitor;
+import org.apache.calcite.sql.util.SqlVisitor;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+import org.apache.calcite.util.Litmus;
+
+import java.util.Arrays;
+
+import static org.apache.calcite.util.Static.RESOURCE;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * The colon operator {@code :}, used for variant path access.
+ *
+ * <p>Operands are {@code (base, path)}, where {@code path} is a
+ * {@link SqlNodeList} of segments.
+ */
+public class SqlColonOperator extends SqlSpecialOperator {
+ SqlColonOperator() {
+ super("COLON", SqlKind.COLON, 100, true, null, null, null);
+ }
+
+ // Path segments are structural literals/identifiers, never column refs, so
+ // they must not be routed through expression visitors such as AggChecker.
+ @Override public <R> void acceptCall(SqlVisitor<R> visitor, SqlCall call,
+ boolean onlyExpressions, SqlBasicVisitor.ArgHandler<R> argHandler) {
+ if (onlyExpressions) {
+ argHandler.visitChild(visitor, call, 0, call.operand(0));
+ } else {
+ super.acceptCall(visitor, call, onlyExpressions, argHandler);
+ }
+ }
+
+ @Override public ReduceResult reduceExpr(int ordinal, TokenSequence list) {
+ final SqlNode left = list.node(ordinal - 1);
+ final SqlNode right = list.node(ordinal + 1);
+ return new ReduceResult(ordinal - 1, ordinal + 2,
+ createCall(
+ SqlParserPos.sum(
+ Arrays.asList(
+ requireNonNull(left, "left").getParserPosition(),
+ requireNonNull(right, "right").getParserPosition(),
+ list.pos(ordinal))),
+ left,
+ right));
+ }
+
+ @Override public void unparse(SqlWriter writer, SqlCall call, int leftPrec,
+ int rightPrec) {
+ final SqlWriter.Frame frame =
+ writer.startList(SqlWriter.FrameTypeEnum.IDENTIFIER);
+ call.operand(0).unparse(writer, leftPrec, 0);
+ writer.setNeedWhitespace(false);
+ writer.keyword(":");
+ writer.setNeedWhitespace(false);
+ boolean first = true;
+ for (SqlNode segment : (SqlNodeList) call.operand(1)) {
+ if (segment instanceof SqlIdentifier) {
+ if (!first) {
+ writer.sep(".");
+ }
+ segment.unparse(writer, 0, 0);
+ } else {
+ final SqlWriter.Frame b = writer.startList("[", "]");
+ segment.unparse(writer, 0, 0);
+ writer.endList(b);
+ }
+ first = false;
+ }
+ writer.endList(frame);
+ }
+
+ @Override public SqlOperandCountRange getOperandCountRange() {
+ return SqlOperandCountRanges.of(2);
+ }
+
+ @Override public RelDataType deriveType(SqlValidator validator,
+ SqlValidatorScope scope, SqlCall call) {
+ final RelDataType baseType =
+ requireNonNull(validator.deriveType(scope, call.operand(0)));
+ final SqlTypeName name = baseType.getSqlTypeName();
+ if (name != SqlTypeName.VARIANT && name != SqlTypeName.ANY) {
Review Comment:
Good point, created `OperandTypes.COLON`.
On STRING handling, VARIANT is intentional for this PR. The API is kept simple enough that STRING can be added later by extending the existing operand checker and convertlet. Happy to do this in a follow-up Jira.
##########
core/src/main/java/org/apache/calcite/sql/fun/SqlColonOperator.java:
##########
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlSpecialOperator;
+import org.apache.calcite.sql.SqlUtil;
+import org.apache.calcite.sql.SqlWriter;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.sql.util.SqlBasicVisitor;
+import org.apache.calcite.sql.util.SqlVisitor;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+import org.apache.calcite.util.Litmus;
+
+import java.util.Arrays;
+
+import static org.apache.calcite.util.Static.RESOURCE;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * The colon operator {@code :}, used for variant path access.
+ *
+ * <p>Operands are {@code (base, path)}, where {@code path} is a
+ * {@link SqlNodeList} of segments.
+ */
+public class SqlColonOperator extends SqlSpecialOperator {
+ SqlColonOperator() {
+ super("COLON", SqlKind.COLON, 100, true, null, null, null);
+ }
+
+ // Path segments are structural literals/identifiers, never column refs, so
+ // they must not be routed through expression visitors such as AggChecker.
+ @Override public <R> void acceptCall(SqlVisitor<R> visitor, SqlCall call,
+ boolean onlyExpressions, SqlBasicVisitor.ArgHandler<R> argHandler) {
+ if (onlyExpressions) {
+ argHandler.visitChild(visitor, call, 0, call.operand(0));
+ } else {
+ super.acceptCall(visitor, call, onlyExpressions, argHandler);
+ }
+ }
+
+ @Override public ReduceResult reduceExpr(int ordinal, TokenSequence list) {
+ final SqlNode left = list.node(ordinal - 1);
+ final SqlNode right = list.node(ordinal + 1);
+ return new ReduceResult(ordinal - 1, ordinal + 2,
+ createCall(
+ SqlParserPos.sum(
+ Arrays.asList(
+ requireNonNull(left, "left").getParserPosition(),
+ requireNonNull(right, "right").getParserPosition(),
+ list.pos(ordinal))),
+ left,
+ right));
+ }
+
+ @Override public void unparse(SqlWriter writer, SqlCall call, int leftPrec,
+ int rightPrec) {
+ final SqlWriter.Frame frame =
+ writer.startList(SqlWriter.FrameTypeEnum.IDENTIFIER);
+ call.operand(0).unparse(writer, leftPrec, 0);
+ writer.setNeedWhitespace(false);
+ writer.keyword(":");
+ writer.setNeedWhitespace(false);
+ boolean first = true;
+ for (SqlNode segment : (SqlNodeList) call.operand(1)) {
+ if (segment instanceof SqlIdentifier) {
+ if (!first) {
+ writer.sep(".");
+ }
+ segment.unparse(writer, 0, 0);
+ } else {
+ final SqlWriter.Frame b = writer.startList("[", "]");
+ segment.unparse(writer, 0, 0);
+ writer.endList(b);
+ }
+ first = false;
+ }
+ writer.endList(frame);
+ }
+
+ @Override public SqlOperandCountRange getOperandCountRange() {
+ return SqlOperandCountRanges.of(2);
+ }
+
+ @Override public RelDataType deriveType(SqlValidator validator,
+ SqlValidatorScope scope, SqlCall call) {
+ final RelDataType baseType =
+ requireNonNull(validator.deriveType(scope, call.operand(0)));
+ final SqlTypeName name = baseType.getSqlTypeName();
+ if (name != SqlTypeName.VARIANT && name != SqlTypeName.ANY) {
+ throw SqlUtil.newContextException(call.operand(0).getParserPosition(),
+ RESOURCE.incompatibleTypes());
+ }
+ final RelDataType type =
+ validator.getTypeFactory().createTypeWithNullability(
+ validator.getTypeFactory().createSqlType(SqlTypeName.VARIANT), true);
+ validator.setValidatedNodeType(call, type);
+ return type;
+ }
+
+ @Override public void validateCall(SqlCall call, SqlValidator validator,
+ SqlValidatorScope scope, SqlValidatorScope operandScope) {
+ assert call.getOperator() == this;
+ call.operand(0).validateExpr(validator, operandScope);
+ }
+
+ @Override public boolean validRexOperands(final int count, final Litmus litmus) {
+ return litmus.fail("COLON is valid only for SqlCall not for RexCall");
+ }
+
+ @Override public String getAllowedSignatures(String name) {
+ return "<A>:<PATH>";
Review Comment:
Whops, fixed it, also moved this part to `OperandTypes`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add parser support for ':' field/item access syntax [calcite]
Posted by "asolimando (via GitHub)" <gi...@apache.org>.
asolimando commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4293792033
> But isn't the right solution to first introduce a colon operator and just parse into that?
IMO it's understandable, as a new contributor, to try and keep the changes minimal (or whatever will decrease friction), but since we are happy with having the final solution from the beginning, no need for doing it in two steps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add parser support for ':' field/item access syntax [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3120233492
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -3774,6 +3774,48 @@ SqlNode Expression(ExprContext exprContext) :
list = Expression2(exprContext) { return SqlParserUtil.toTree(list); }
}
+/** Parses regular postfixes such as {@code a.b}, {@code a[1]} and
+ * {@code a[1].b}. Shared by the main expression grammar and Babel's
+ * infix cast. */
+void AddRegularPostfixes(List<Object> list) :
+{
+ SqlNode ext;
+}
+{
+ (
+ LOOKAHEAD(2) <DOT>
+ ext = RowExpressionExtension() {
+ list.add(
+ new SqlParserUtil.ToTreeListItem(
+ SqlStdOperatorTable.DOT, getPos()));
+ list.add(ext);
+ }
+ |
+ AddBracketPostfix(list)
+ )*
+}
+
+/** Parses postfixes that are allowed after colon field access, for example
+ * {@code v:field.nested} or {@code v:[1]['x']}. Unlike regular postfixes,
+ * this path does not allow member-style calls such as {@code v:field.func()}. */
+void AddColonPostfixes(List<Object> list) :
+{
+ SqlIdentifier p;
+}
+{
+ (
+ LOOKAHEAD(2) <DOT>
+ p = SimpleIdentifier() {
+ list.add(
+ new SqlParserUtil.ToTreeListItem(
Review Comment:
So is the plan in a subsequent PR to add a Colon operator and replace this with colon?
Then I would suggest using the colon from the start. You don't want people to take a dependence on this parse tree and to expect a dot. Why change something if you can do it right from the start?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3171278126
##########
core/src/main/java/org/apache/calcite/sql/fun/SqlColonOperator.java:
##########
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlSpecialOperator;
+import org.apache.calcite.sql.SqlUtil;
+import org.apache.calcite.sql.SqlWriter;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.sql.util.SqlBasicVisitor;
+import org.apache.calcite.sql.util.SqlVisitor;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+import org.apache.calcite.util.Litmus;
+
+import java.util.Arrays;
+
+import static org.apache.calcite.util.Static.RESOURCE;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * The colon operator {@code :}, used for variant path access.
+ *
+ * <p>Operands are {@code (base, path)}, where {@code path} is a
+ * {@link SqlNodeList} of segments.
+ */
+public class SqlColonOperator extends SqlSpecialOperator {
+ SqlColonOperator() {
+ super("COLON", SqlKind.COLON, 100, true, null, null, null);
+ }
+
+ // Path segments are structural literals/identifiers, never column refs, so
+ // they must not be routed through expression visitors such as AggChecker.
+ @Override public <R> void acceptCall(SqlVisitor<R> visitor, SqlCall call,
+ boolean onlyExpressions, SqlBasicVisitor.ArgHandler<R> argHandler) {
+ if (onlyExpressions) {
+ argHandler.visitChild(visitor, call, 0, call.operand(0));
+ } else {
+ super.acceptCall(visitor, call, onlyExpressions, argHandler);
+ }
+ }
+
+ @Override public ReduceResult reduceExpr(int ordinal, TokenSequence list) {
+ final SqlNode left = list.node(ordinal - 1);
+ final SqlNode right = list.node(ordinal + 1);
+ return new ReduceResult(ordinal - 1, ordinal + 2,
+ createCall(
+ SqlParserPos.sum(
+ Arrays.asList(
+ requireNonNull(left, "left").getParserPosition(),
+ requireNonNull(right, "right").getParserPosition(),
+ list.pos(ordinal))),
+ left,
+ right));
+ }
+
+ @Override public void unparse(SqlWriter writer, SqlCall call, int leftPrec,
+ int rightPrec) {
+ final SqlWriter.Frame frame =
+ writer.startList(SqlWriter.FrameTypeEnum.IDENTIFIER);
+ call.operand(0).unparse(writer, leftPrec, 0);
+ writer.setNeedWhitespace(false);
+ writer.keyword(":");
+ writer.setNeedWhitespace(false);
+ boolean first = true;
+ for (SqlNode segment : (SqlNodeList) call.operand(1)) {
+ if (segment instanceof SqlIdentifier) {
+ if (!first) {
+ writer.sep(".");
+ }
+ segment.unparse(writer, 0, 0);
+ } else {
+ final SqlWriter.Frame b = writer.startList("[", "]");
+ segment.unparse(writer, 0, 0);
+ writer.endList(b);
+ }
+ first = false;
+ }
+ writer.endList(frame);
+ }
+
+ @Override public SqlOperandCountRange getOperandCountRange() {
+ return SqlOperandCountRanges.of(2);
+ }
+
+ @Override public RelDataType deriveType(SqlValidator validator,
+ SqlValidatorScope scope, SqlCall call) {
+ final RelDataType baseType =
+ requireNonNull(validator.deriveType(scope, call.operand(0)));
+ final SqlTypeName name = baseType.getSqlTypeName();
+ if (name != SqlTypeName.VARIANT && name != SqlTypeName.ANY) {
+ throw SqlUtil.newContextException(call.operand(0).getParserPosition(),
+ RESOURCE.incompatibleTypes());
+ }
+ final RelDataType type =
+ validator.getTypeFactory().createTypeWithNullability(
+ validator.getTypeFactory().createSqlType(SqlTypeName.VARIANT), true);
+ validator.setValidatedNodeType(call, type);
+ return type;
+ }
+
+ @Override public void validateCall(SqlCall call, SqlValidator validator,
+ SqlValidatorScope scope, SqlValidatorScope operandScope) {
+ assert call.getOperator() == this;
+ call.operand(0).validateExpr(validator, operandScope);
+ }
+
+ @Override public boolean validRexOperands(final int count, final Litmus litmus) {
+ return litmus.fail("COLON is valid only for SqlCall not for RexCall");
+ }
+
+ @Override public String getAllowedSignatures(String name) {
+ return "<A>:<PATH>";
Review Comment:
I don't know what <A> is, it should be a type name.
`<VARIANT>` for sure, `<CHAR>` if you decide that's acceptable too.
##########
core/src/main/java/org/apache/calcite/sql/fun/SqlColonOperator.java:
##########
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlSpecialOperator;
+import org.apache.calcite.sql.SqlUtil;
+import org.apache.calcite.sql.SqlWriter;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.sql.util.SqlBasicVisitor;
+import org.apache.calcite.sql.util.SqlVisitor;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+import org.apache.calcite.util.Litmus;
+
+import java.util.Arrays;
+
+import static org.apache.calcite.util.Static.RESOURCE;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * The colon operator {@code :}, used for variant path access.
+ *
+ * <p>Operands are {@code (base, path)}, where {@code path} is a
+ * {@link SqlNodeList} of segments.
+ */
+public class SqlColonOperator extends SqlSpecialOperator {
+ SqlColonOperator() {
+ super("COLON", SqlKind.COLON, 100, true, null, null, null);
+ }
+
+ // Path segments are structural literals/identifiers, never column refs, so
+ // they must not be routed through expression visitors such as AggChecker.
+ @Override public <R> void acceptCall(SqlVisitor<R> visitor, SqlCall call,
+ boolean onlyExpressions, SqlBasicVisitor.ArgHandler<R> argHandler) {
+ if (onlyExpressions) {
+ argHandler.visitChild(visitor, call, 0, call.operand(0));
+ } else {
+ super.acceptCall(visitor, call, onlyExpressions, argHandler);
+ }
+ }
+
+ @Override public ReduceResult reduceExpr(int ordinal, TokenSequence list) {
+ final SqlNode left = list.node(ordinal - 1);
+ final SqlNode right = list.node(ordinal + 1);
+ return new ReduceResult(ordinal - 1, ordinal + 2,
+ createCall(
+ SqlParserPos.sum(
+ Arrays.asList(
+ requireNonNull(left, "left").getParserPosition(),
+ requireNonNull(right, "right").getParserPosition(),
+ list.pos(ordinal))),
+ left,
+ right));
+ }
+
+ @Override public void unparse(SqlWriter writer, SqlCall call, int leftPrec,
+ int rightPrec) {
+ final SqlWriter.Frame frame =
+ writer.startList(SqlWriter.FrameTypeEnum.IDENTIFIER);
+ call.operand(0).unparse(writer, leftPrec, 0);
+ writer.setNeedWhitespace(false);
+ writer.keyword(":");
+ writer.setNeedWhitespace(false);
+ boolean first = true;
+ for (SqlNode segment : (SqlNodeList) call.operand(1)) {
+ if (segment instanceof SqlIdentifier) {
+ if (!first) {
+ writer.sep(".");
+ }
+ segment.unparse(writer, 0, 0);
+ } else {
+ final SqlWriter.Frame b = writer.startList("[", "]");
+ segment.unparse(writer, 0, 0);
+ writer.endList(b);
+ }
+ first = false;
+ }
+ writer.endList(frame);
+ }
+
+ @Override public SqlOperandCountRange getOperandCountRange() {
+ return SqlOperandCountRanges.of(2);
+ }
+
+ @Override public RelDataType deriveType(SqlValidator validator,
+ SqlValidatorScope scope, SqlCall call) {
+ final RelDataType baseType =
+ requireNonNull(validator.deriveType(scope, call.operand(0)));
+ final SqlTypeName name = baseType.getSqlTypeName();
+ if (name != SqlTypeName.VARIANT && name != SqlTypeName.ANY) {
Review Comment:
Databricks supports string types here too.
It's fine not to support them, but is this your intent?
##########
core/src/main/java/org/apache/calcite/sql/fun/SqlColonOperator.java:
##########
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlSpecialOperator;
+import org.apache.calcite.sql.SqlUtil;
+import org.apache.calcite.sql.SqlWriter;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.sql.util.SqlBasicVisitor;
+import org.apache.calcite.sql.util.SqlVisitor;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+import org.apache.calcite.util.Litmus;
+
+import java.util.Arrays;
+
+import static org.apache.calcite.util.Static.RESOURCE;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * The colon operator {@code :}, used for variant path access.
+ *
+ * <p>Operands are {@code (base, path)}, where {@code path} is a
+ * {@link SqlNodeList} of segments.
+ */
+public class SqlColonOperator extends SqlSpecialOperator {
+ SqlColonOperator() {
+ super("COLON", SqlKind.COLON, 100, true, null, null, null);
+ }
+
+ // Path segments are structural literals/identifiers, never column refs, so
+ // they must not be routed through expression visitors such as AggChecker.
+ @Override public <R> void acceptCall(SqlVisitor<R> visitor, SqlCall call,
+ boolean onlyExpressions, SqlBasicVisitor.ArgHandler<R> argHandler) {
+ if (onlyExpressions) {
+ argHandler.visitChild(visitor, call, 0, call.operand(0));
+ } else {
+ super.acceptCall(visitor, call, onlyExpressions, argHandler);
+ }
+ }
+
+ @Override public ReduceResult reduceExpr(int ordinal, TokenSequence list) {
+ final SqlNode left = list.node(ordinal - 1);
+ final SqlNode right = list.node(ordinal + 1);
+ return new ReduceResult(ordinal - 1, ordinal + 2,
+ createCall(
+ SqlParserPos.sum(
+ Arrays.asList(
+ requireNonNull(left, "left").getParserPosition(),
+ requireNonNull(right, "right").getParserPosition(),
+ list.pos(ordinal))),
+ left,
+ right));
+ }
+
+ @Override public void unparse(SqlWriter writer, SqlCall call, int leftPrec,
+ int rightPrec) {
+ final SqlWriter.Frame frame =
+ writer.startList(SqlWriter.FrameTypeEnum.IDENTIFIER);
+ call.operand(0).unparse(writer, leftPrec, 0);
+ writer.setNeedWhitespace(false);
+ writer.keyword(":");
+ writer.setNeedWhitespace(false);
+ boolean first = true;
+ for (SqlNode segment : (SqlNodeList) call.operand(1)) {
+ if (segment instanceof SqlIdentifier) {
+ if (!first) {
+ writer.sep(".");
+ }
+ segment.unparse(writer, 0, 0);
+ } else {
+ final SqlWriter.Frame b = writer.startList("[", "]");
+ segment.unparse(writer, 0, 0);
+ writer.endList(b);
+ }
+ first = false;
+ }
+ writer.endList(frame);
+ }
+
+ @Override public SqlOperandCountRange getOperandCountRange() {
+ return SqlOperandCountRanges.of(2);
+ }
+
+ @Override public RelDataType deriveType(SqlValidator validator,
+ SqlValidatorScope scope, SqlCall call) {
+ final RelDataType baseType =
+ requireNonNull(validator.deriveType(scope, call.operand(0)));
+ final SqlTypeName name = baseType.getSqlTypeName();
+ if (name != SqlTypeName.VARIANT && name != SqlTypeName.ANY) {
Review Comment:
It looks like some of this code belongs in operandTypeChecker instead.
The return type is just the operand 0 type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4169446703
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "caicancai (via GitHub)" <gi...@apache.org>.
caicancai commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4199738854
@tmater Thank you for following up. I left a question on Jira. Could you answer it when you have time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3198983164
##########
core/src/main/java/org/apache/calcite/sql/type/OperandTypes.java:
##########
@@ -554,6 +554,38 @@ public static SqlOperandTypeChecker variadic(
.or(OperandTypes.family(SqlTypeFamily.VARIANT))
.or(OperandTypes.family(SqlTypeFamily.ANY));
+ /**
+ * Operand type-checking strategy for the colon operator: first operand
+ * must be {@code VARIANT} (or {@code ANY}); the second operand is a
+ * path expression and is not type-checked.
+ */
+ public static final SqlOperandTypeChecker COLON =
Review Comment:
Is this code ever exercised if the convertlet table replaces this?
I.e., does it have coverage in your tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4404134128
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [5 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [96.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3082134691
##########
babel/src/test/java/org/apache/calcite/test/BabelParserTest.java:
##########
@@ -301,6 +306,101 @@ private void checkParseInfixCast(String sqlType) {
sql(sql).ok(expected);
}
+ @Test void testColonFieldAccessWithInfixCast() {
+ final SqlParserFixture f =
+ fixture().withConformance(new SqlAbstractConformance() {
+ @Override public boolean isColonFieldAccessAllowed() {
+ return true;
+ }
+ });
+
+ // Bracket after :: binds to the type, not as subscript on the cast result
+ sql("select v::varchar array[1] from t")
+ .ok("SELECT `V` :: VARCHAR ARRAY[1]\nFROM `T`");
+ f.sql("select v::varchar array[1] from t")
Review Comment:
I still don't understand what the parse tree of this expression is.
I think the parser should actually reject this expression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4126967143
> I don't understand Calcite JavaCC code very well, I might need to learn it. If no one reviews it this week, I'll start working on it next week.
Thank you for taking a look, @caicancai. I agree this is still a fairly complex PR, and I wouldn’t claim to fully understand all of its implications yet.
That’s also why I tried to include a wide range of test variations and keep the changes gated behind a conformance config.
I’ve organized it into five commits (prior to the CI failures) to make the review easier. If it would help further, I’m happy to split it into 4–5 smaller PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4280129916
> Can you even parse the expression if you don't have a colon operator?
Your other issue about introducing it seems fine.
Yes, it is usable as-is. The current parser implementation lowers `:` access into the normal `DOT` / `ITEM` AST shape, this unblocks the syntax problems and does not break anything. Then, I do a post-parse rewrite pass that re-identifies colon-origin access and rewrites those calls into dedicated colon operators, so later planner stages can still tell them apart. It is not clean, but a great first milestone.
I chose this shape intentionally to keep the initial upstream parser change small and unblock the syntax problem first, rather than mixing it with a broader AST/operator change in the same step. I’d be happy to follow up with adding the operator as a next step.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3076409987
##########
babel/src/test/java/org/apache/calcite/test/BabelParserTest.java:
##########
@@ -301,6 +306,101 @@ private void checkParseInfixCast(String sqlType) {
sql(sql).ok(expected);
}
+ @Test void testColonFieldAccessWithInfixCast() {
+ final SqlParserFixture f =
+ fixture().withConformance(new SqlAbstractConformance() {
+ @Override public boolean isColonFieldAccessAllowed() {
+ return true;
+ }
+ });
+
+ // Bracket after :: binds to the type, not as subscript on the cast result
+ sql("select v::varchar[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+ f.sql("select v::varchar[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+
+ // Explicit parentheses make no difference — bracket still binds to the type
+ sql("select (v::varchar)[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+ f.sql("select (v::varchar)[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+
+ // Array type with bracket
+ sql("select v::integer array[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+ f.sql("select v::integer array[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+
+ // Parenthesized array type with bracket
+ sql("select (v::integer array)[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+ f.sql("select (v::integer array)[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+
+ // Multiple brackets bind to the type
+ sql("select v::varchar[1][2] from t")
+ .ok("SELECT `V` :: VARCHAR[1][2]\nFROM `T`");
Review Comment:
But `VARCHAR[1]` is not a type. What does this mean?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' field/item access syntax [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3158251190
##########
core/src/main/java/org/apache/calcite/sql/fun/SqlColonOperator.java:
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlSpecialOperator;
+import org.apache.calcite.sql.SqlWriter;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.sql.util.SqlBasicVisitor;
+import org.apache.calcite.sql.util.SqlVisitor;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+import org.apache.calcite.util.Litmus;
+
+import java.util.Arrays;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * The colon operator {@code :}, which preserves field/item path access syntax
+ * in the SQL tree.
+ *
+ * <p>Operands are {@code (base, path)}, where {@code path} is a
+ * {@link SqlNodeList} of structural segments only: identifiers (dot-notation),
+ * character literals (bracketed string keys), or numeric literals (bracketed
+ * indexes).
+ *
+ * <p>Calcite does not lower this operator to Rex. Engines must register a
Review Comment:
Frankly, the way it's used is not the business of the class.
The JavaDoc should describe just what this class does, not what other classes do with it. If the other classes change, this comment will become obsolete.
This comment should probably be in the convertlet table if at all.
##########
core/src/main/java/org/apache/calcite/sql/fun/SqlColonOperator.java:
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlSpecialOperator;
+import org.apache.calcite.sql.SqlWriter;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.sql.util.SqlBasicVisitor;
+import org.apache.calcite.sql.util.SqlVisitor;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+import org.apache.calcite.util.Litmus;
+
+import java.util.Arrays;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * The colon operator {@code :}, which preserves field/item path access syntax
+ * in the SQL tree.
+ *
+ * <p>Operands are {@code (base, path)}, where {@code path} is a
+ * {@link SqlNodeList} of structural segments only: identifiers (dot-notation),
+ * character literals (bracketed string keys), or numeric literals (bracketed
+ * indexes).
+ *
+ * <p>Calcite does not lower this operator to Rex. Engines must register a
+ * convertlet for {@link SqlKind#COLON}; without one, sql2rel fails.
+ */
+public class SqlColonOperator extends SqlSpecialOperator {
+ SqlColonOperator() {
+ super("COLON", SqlKind.COLON, 100, true, null, null, null);
+ }
+
+ // Path segments are structural literals/identifiers, never column refs, so
+ // they must not be routed through expression visitors such as AggChecker.
+ @Override public <R> void acceptCall(SqlVisitor<R> visitor, SqlCall call,
+ boolean onlyExpressions, SqlBasicVisitor.ArgHandler<R> argHandler) {
+ if (onlyExpressions) {
+ argHandler.visitChild(visitor, call, 0, call.operand(0));
+ } else {
+ super.acceptCall(visitor, call, onlyExpressions, argHandler);
+ }
+ }
+
+ @Override public ReduceResult reduceExpr(int ordinal, TokenSequence list) {
+ final SqlNode left = list.node(ordinal - 1);
+ final SqlNode right = list.node(ordinal + 1);
+ return new ReduceResult(ordinal - 1, ordinal + 2,
+ createCall(
+ SqlParserPos.sum(
+ Arrays.asList(
+ requireNonNull(left, "left").getParserPosition(),
+ requireNonNull(right, "right").getParserPosition(),
+ list.pos(ordinal))),
+ left,
+ right));
+ }
+
+ @Override public void unparse(SqlWriter writer, SqlCall call, int leftPrec,
+ int rightPrec) {
+ final SqlWriter.Frame frame =
+ writer.startList(SqlWriter.FrameTypeEnum.IDENTIFIER);
+ call.operand(0).unparse(writer, leftPrec, 0);
+ writer.setNeedWhitespace(false);
+ writer.keyword(":");
+ writer.setNeedWhitespace(false);
+ boolean first = true;
+ for (SqlNode segment : (SqlNodeList) call.operand(1)) {
+ if (segment instanceof SqlIdentifier) {
+ if (!first) {
+ writer.sep(".");
+ }
+ segment.unparse(writer, 0, 0);
+ } else {
+ final SqlWriter.Frame b = writer.startList("[", "]");
+ segment.unparse(writer, 0, 0);
+ writer.endList(b);
+ }
+ first = false;
+ }
+ writer.endList(frame);
+ }
+
+ @Override public SqlOperandCountRange getOperandCountRange() {
+ return SqlOperandCountRanges.of(2);
+ }
+
+ // Returns nullable ANY: path access can yield NULL on a missing key, and
+ // the concrete type is up to the engine's convertlet.
Review Comment:
You again assume something about the convertlet table.
This type seems overly general, can't you compute the type of a colon expression from the types of its operands?
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlConformance.java:
##########
@@ -260,6 +260,21 @@ enum SelectAliasLookup {
*/
boolean allowHyphenInUnquotedTableName();
+ /**
+ * Whether {@code :} is allowed as a field/item access operator.
+ *
+ * <p>If true, expressions such as {@code v:field} and {@code v:['field']}
+ * are valid. In this mode, JSON constructors must use the
+ * {@code KEY ... VALUE} form rather than {@code :} or comma-pair syntax.
+ *
+ * <p>No built-in conformance level enables this; engines that want
Review Comment:
Again you describe in JavaDoc what other classes do.
I think you should only refer to what is happening here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4169739039
> Is the behavior with `::` needing parenthesis a new feature, or was this already present previously?
@mihaibudiu , good catch! The parenthesis requirement itself is not new, on `main`, `v::varchar[1]` already means "cast to `VARCHAR[1]`" (bracket binds to the type), so parentheses were always needed to subscript the cast result.
However, you're right that something was off: the bracket refactor broke parsing `v::varchar[1]` entirely, because InfixCast was implicitly relying on `Expression2()`'s loop to consume postfixes. This commit fixes that by adding an explicit `AddRegularPostfixes(list)` call after `DataType()` in `InfixCast`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4124874429
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [99.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4128362649
Is the behavior with `::` needing parenthesis a new feature, or was this already present previously?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3088124236
##########
babel/src/test/java/org/apache/calcite/test/BabelParserTest.java:
##########
@@ -301,6 +306,101 @@ private void checkParseInfixCast(String sqlType) {
sql(sql).ok(expected);
}
+ @Test void testColonFieldAccessWithInfixCast() {
+ final SqlParserFixture f =
+ fixture().withConformance(new SqlAbstractConformance() {
+ @Override public boolean isColonFieldAccessAllowed() {
+ return true;
+ }
+ });
+
+ // Bracket after :: binds to the type, not as subscript on the cast result
+ sql("select v::varchar array[1] from t")
+ .ok("SELECT `V` :: VARCHAR ARRAY[1]\nFROM `T`");
+ f.sql("select v::varchar array[1] from t")
Review Comment:
Ok, if the behavior does not change, it's fine. If we think there's a bug in the parser, where it produces a nonsensical parse tree, we should probably file an issue which can be solved separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4233596688
I was on vacation, I plan to review this PR again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4202679244
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4269421934
Can you even parse the expression if you don't have a colon operator?
Your other issue about introducing it seems fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4268669015
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [3 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "sbroeder (via GitHub)" <gi...@apache.org>.
sbroeder commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4127640108
Have you created a Calcite artifact and created a Dremio PR to see if there are unwanted side effects?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add parser support for ':' field/item access syntax [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4335502483
Thanks for the review and sorry for the delay, @mihaibudiu and @asolimando.
Yes, I wanted to keep the change small and build the feature iteratively. It helps me focus on small milestones, and I think it also makes the review easier.
The branch was a month old, so I figured a rebase was overdue, especially given the direction change.
The design is no longer a simple rewrite of `COLON` into `DOT`/`ITEM`. I reverted the parser refactors that were needed for that, added parsing for the `COLON` operator directly, and introduced a dedicated `SqlColonOperator` that engines can lower via their own convertlet (e.g. to `VARIANT_GET`).
PTAL when you have a moment, happy to iterate further.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu merged PR #4844:
URL: https://github.com/apache/calcite/pull/4844
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4403408239
Please squash the commits to a single one; use the same title for the commit, JIRA title and PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3199338846
##########
core/src/main/java/org/apache/calcite/sql/type/OperandTypes.java:
##########
@@ -554,6 +554,38 @@ public static SqlOperandTypeChecker variadic(
.or(OperandTypes.family(SqlTypeFamily.VARIANT))
.or(OperandTypes.family(SqlTypeFamily.ANY));
+ /**
+ * Operand type-checking strategy for the colon operator: first operand
+ * must be {@code VARIANT} (or {@code ANY}); the second operand is a
+ * path expression and is not type-checked.
+ */
+ public static final SqlOperandTypeChecker COLON =
Review Comment:
Yes, validation runs before `SqlToRelConverter`, so by the time the convertlet could rewrite it to `ITEM`, the type check has already failed. The `skill:type` failing case in `SqlValidatorTest.java:1546` exercises it directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4395347748
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [5 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [96.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3079319816
##########
babel/src/test/java/org/apache/calcite/test/BabelParserTest.java:
##########
@@ -301,6 +306,101 @@ private void checkParseInfixCast(String sqlType) {
sql(sql).ok(expected);
}
+ @Test void testColonFieldAccessWithInfixCast() {
+ final SqlParserFixture f =
+ fixture().withConformance(new SqlAbstractConformance() {
+ @Override public boolean isColonFieldAccessAllowed() {
+ return true;
+ }
+ });
+
+ // Bracket after :: binds to the type, not as subscript on the cast result
+ sql("select v::varchar[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+ f.sql("select v::varchar[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+
+ // Explicit parentheses make no difference — bracket still binds to the type
+ sql("select (v::varchar)[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+ f.sql("select (v::varchar)[1] from t")
+ .ok("SELECT `V` :: VARCHAR[1]\nFROM `T`");
+
+ // Array type with bracket
+ sql("select v::integer array[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+ f.sql("select v::integer array[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+
+ // Parenthesized array type with bracket
+ sql("select (v::integer array)[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+ f.sql("select (v::integer array)[1] from t")
+ .ok("SELECT `V` :: INTEGER ARRAY[1]\nFROM `T`");
+
+ // Multiple brackets bind to the type
+ sql("select v::varchar[1][2] from t")
+ .ok("SELECT `V` :: VARCHAR[1][2]\nFROM `T`");
Review Comment:
This is parser coverage rather than a meaningful type assertion. The test is checking that after `::`, the parser still accepts postfix syntax on the RHS and groups it there, rather than attaching the postfix to the cast result. I agree the old example made that intent unclear, so I updated the test to make the associativity point clearer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3088234553
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -3791,13 +3809,61 @@ void AddExpression2b(List<Object> list, ExprContext exprContext) :
e = Expression3(exprContext) {
list.add(e);
}
+ AddRegularPostfixes(list)
+ [
+ LOOKAHEAD(2, <COLON> SimpleIdentifier(),
+ { this.conformance.isColonFieldAccessAllowed() })
+ <COLON>
+ ext = SimpleIdentifier() {
+ list.add(
+ new SqlParserUtil.ToTreeListItem(
+ SqlStdOperatorTable.DOT, getPos()));
+ list.add(ext);
+ }
+ AddRegularPostfixes(list)
+ |
+ LOOKAHEAD(2, <COLON> <LBRACKET>,
+ { this.conformance.isColonFieldAccessAllowed() })
+ <COLON>
+ AddBracketAccess(list)
+ AddRegularPostfixes(list)
+ ]
+}
+
+void AddBracketAccess(List<Object> list) :
Review Comment:
please add a comment describing what this is expected to parse; I know this code has been moved, but since the context of the caller is missing, the comment will help maintainers.
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -3791,13 +3809,61 @@ void AddExpression2b(List<Object> list, ExprContext exprContext) :
e = Expression3(exprContext) {
list.add(e);
}
+ AddRegularPostfixes(list)
+ [
+ LOOKAHEAD(2, <COLON> SimpleIdentifier(),
+ { this.conformance.isColonFieldAccessAllowed() })
+ <COLON>
Review Comment:
Does this cover the Databricks and Snowflake capabilities?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3085805331
##########
babel/src/test/java/org/apache/calcite/test/BabelParserTest.java:
##########
@@ -301,6 +306,101 @@ private void checkParseInfixCast(String sqlType) {
sql(sql).ok(expected);
}
+ @Test void testColonFieldAccessWithInfixCast() {
+ final SqlParserFixture f =
+ fixture().withConformance(new SqlAbstractConformance() {
+ @Override public boolean isColonFieldAccessAllowed() {
+ return true;
+ }
+ });
+
+ // Bracket after :: binds to the type, not as subscript on the cast result
+ sql("select v::varchar array[1] from t")
+ .ok("SELECT `V` :: VARCHAR ARRAY[1]\nFROM `T`");
+ f.sql("select v::varchar array[1] from t")
Review Comment:
I see your point, this is quite ambiguous. The reason I added these cases was mainly to show that we did not introduce regressions while touching the `INFIX_CAST` grammar.
For `v::varchar array[1]`, the tree is something like:
```
INFIX_CAST(
v,
ITEM(SqlDataTypeSpec(VARCHAR ARRAY), 1)
)
```
So the `[1]` is attaching on the type side, not as a subscript on the cast result.
Also, I backported these tests onto the base of this PR as proof: [#4885](https://github.com/apache/calcite/pull/4885). The same behavior is already present there, so this is not something introduced by the colon-field-access change.
Would you prefer fixing this ambiguity in this PR, or filing a Jira to clean it up separately? I am leaning towards a separate PR and polishing these tests a bit, maybe leaving only one ambiguous expression test just in case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "caicancai (via GitHub)" <gi...@apache.org>.
caicancai commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4122699229
I might wait until the CI issue is resolved before reviewing this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "caicancai (via GitHub)" <gi...@apache.org>.
caicancai commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4126489164
I don't understand Calcite JavaCC code very well, I might need to learn it. If no one reviews it this week, I'll start working on it next week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4125121995
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for `:` as a field/item access operator [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4285324394
But isn't the right solution to first introduce a colon operator and just parse into that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add parser support for ':' field/item access syntax [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4286442259
> But isn't the right solution to first introduce a colon operator and just parse into that?
I think that is a reasonable follow-up direction, but this PR is intentionally parser-scoped: accept `:` syntax behind conformance, map it to existing `DOT` / `ITEM`, and avoid widening the patch into a broader operator change. I updated the title/description to make that boundary explicit. If you see a first-class colon operator as a prerequisite for this PR, let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' field/item access syntax [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4335601684
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [4 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [97.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4372205316
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844) **Quality Gate passed**
Issues
 [4 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4844&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4844&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [96.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4844&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4844)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "tmater (via GitHub)" <gi...@apache.org>.
tmater commented on PR #4844:
URL: https://github.com/apache/calcite/pull/4844#issuecomment-4404221866
Thanks @mihaibudiu, squashed and rebased.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-7448] Add support for ':' variant path access syntax [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4844:
URL: https://github.com/apache/calcite/pull/4844#discussion_r3198983164
##########
core/src/main/java/org/apache/calcite/sql/type/OperandTypes.java:
##########
@@ -554,6 +554,38 @@ public static SqlOperandTypeChecker variadic(
.or(OperandTypes.family(SqlTypeFamily.VARIANT))
.or(OperandTypes.family(SqlTypeFamily.ANY));
+ /**
+ * Operand type-checking strategy for the colon operator: first operand
+ * must be {@code VARIANT} (or {@code ANY}); the second operand is a
+ * path expression and is not type-checked.
+ */
+ public static final SqlOperandTypeChecker COLON =
Review Comment:
Is this code ever exercised if the convertlet table replaces this?
I.e., does it have coverage in your tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org