You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "clintropolis (via GitHub)" <gi...@apache.org> on 2023/03/07 08:56:37 UTC
[GitHub] [druid] clintropolis opened a new pull request, #13890: use Calcites.getColumnTypeForRelDataType for SQL CAST operator conversion
clintropolis opened a new pull request, #13890:
URL: https://github.com/apache/druid/pull/13890
### Description
This PR modifies the SQL `CAST` operator conversion to use `Calcites.getColumnTypeForRelDataType` to convert calcite types to native Druid types instead of using its own custom `SqlTypeName ` to `ExprType` mapping to make it more consistent with other SQL to Druid type conversions done for most other operators. This allows it to handle some additional casts which were not previously supported, such as those with ARRAY types. The test added to `CalciteMultiValueStringQueryTest` was not able to be planned prior to the changes to the `CAST` operator since during planning explicit casts would be added which would then fail since the mapping table was missing arrays.
Switching `CAST` to use `Calcites.getColumnTypeForRelDataType` also required making a small adjustment to this method to be able to handle `SqlTypeName.DAY_INTERVAL_TYPES` and `SqlTypeName.YEAR_INTERVAL_TYPES` as a Druid `LONG` type. This also allows some additional expressions to be planned that were not previously supported, shown in the changes to `ExpressionsTest`, `GreatestExpressionTest`, and `LeastExpressionTest`.
<hr>
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] a release note entry in the PR description.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a diff in pull request #13890: use Calcites.getColumnTypeForRelDataType for SQL CAST operator conversion
Posted by "clintropolis (via GitHub)" <gi...@apache.org>.
clintropolis commented on code in PR #13890:
URL: https://github.com/apache/druid/pull/13890#discussion_r1127906648
##########
sql/src/test/java/org/apache/druid/sql/calcite/expression/ExpressionsTest.java:
##########
@@ -1779,8 +1778,7 @@ public void testTimeMinusYearMonthInterval()
DruidExpression.functionCall("timestamp_shift"),
ImmutableList.of(
DruidExpression.ofColumn(ColumnType.LONG, "t"),
- // RexNode type "interval year to month" is not reported as ColumnType.STRING
- DruidExpression.ofLiteral(null, DruidExpression.stringLiteral("P13M")),
+ DruidExpression.ofLiteral(ColumnType.LONG, DruidExpression.stringLiteral("P13M")),
Review Comment:
ah, i see why it was treating the cast as LONG typed...
`timestampDiff` test for example starts spitting out explicit casts which are not necessary so I think i need to add a bit extra logic to cast to let intervals cast to longs without a cast ```ExpressionVirtualColumn{name='v0', expression='div(CAST(("__time" - 915148800000), 'LONG'),86400000)', outputType=LONG}```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm merged pull request #13890: use Calcites.getColumnTypeForRelDataType for SQL CAST operator conversion
Posted by "gianm (via GitHub)" <gi...@apache.org>.
gianm merged PR #13890:
URL: https://github.com/apache/druid/pull/13890
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm commented on a diff in pull request #13890: use Calcites.getColumnTypeForRelDataType for SQL CAST operator conversion
Posted by "gianm (via GitHub)" <gi...@apache.org>.
gianm commented on code in PR #13890:
URL: https://github.com/apache/druid/pull/13890#discussion_r1127865062
##########
sql/src/test/java/org/apache/druid/sql/calcite/expression/ExpressionsTest.java:
##########
@@ -1779,8 +1778,7 @@ public void testTimeMinusYearMonthInterval()
DruidExpression.functionCall("timestamp_shift"),
ImmutableList.of(
DruidExpression.ofColumn(ColumnType.LONG, "t"),
- // RexNode type "interval year to month" is not reported as ColumnType.STRING
- DruidExpression.ofLiteral(null, DruidExpression.stringLiteral("P13M")),
+ DruidExpression.ofLiteral(ColumnType.LONG, DruidExpression.stringLiteral("P13M")),
Review Comment:
Long type for string literal? Seems not right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a diff in pull request #13890: use Calcites.getColumnTypeForRelDataType for SQL CAST operator conversion
Posted by "clintropolis (via GitHub)" <gi...@apache.org>.
clintropolis commented on code in PR #13890:
URL: https://github.com/apache/druid/pull/13890#discussion_r1127934496
##########
sql/src/test/java/org/apache/druid/sql/calcite/expression/ExpressionsTest.java:
##########
@@ -1779,8 +1778,7 @@ public void testTimeMinusYearMonthInterval()
DruidExpression.functionCall("timestamp_shift"),
ImmutableList.of(
DruidExpression.ofColumn(ColumnType.LONG, "t"),
- // RexNode type "interval year to month" is not reported as ColumnType.STRING
- DruidExpression.ofLiteral(null, DruidExpression.stringLiteral("P13M")),
+ DruidExpression.ofLiteral(ColumnType.LONG, DruidExpression.stringLiteral("P13M")),
Review Comment:
ok, updated to handle intervals as STRINGS so they match the argument literals in plans, but also handle them as longs for casting (and also reduction). I suppose instead I could just drop intervals from being handled by `Calcites.getColumnTypeForRelDataType` like it was before and just handle them in the cast, but string seems the most correct i think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a diff in pull request #13890: use Calcites.getColumnTypeForRelDataType for SQL CAST operator conversion
Posted by "clintropolis (via GitHub)" <gi...@apache.org>.
clintropolis commented on code in PR #13890:
URL: https://github.com/apache/druid/pull/13890#discussion_r1127934496
##########
sql/src/test/java/org/apache/druid/sql/calcite/expression/ExpressionsTest.java:
##########
@@ -1779,8 +1778,7 @@ public void testTimeMinusYearMonthInterval()
DruidExpression.functionCall("timestamp_shift"),
ImmutableList.of(
DruidExpression.ofColumn(ColumnType.LONG, "t"),
- // RexNode type "interval year to month" is not reported as ColumnType.STRING
- DruidExpression.ofLiteral(null, DruidExpression.stringLiteral("P13M")),
+ DruidExpression.ofLiteral(ColumnType.LONG, DruidExpression.stringLiteral("P13M")),
Review Comment:
ok, updated to handle intervals as STRINGS so they match the argument literals in plans, but also handle them as longs for casting (and also reduction). I suppose instead I could just drop intervals from being handled by `Calcites.getColumnTypeForRelDataType` like it was before, but string seems the most correct i think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a diff in pull request #13890: use Calcites.getColumnTypeForRelDataType for SQL CAST operator conversion
Posted by "clintropolis (via GitHub)" <gi...@apache.org>.
clintropolis commented on code in PR #13890:
URL: https://github.com/apache/druid/pull/13890#discussion_r1127900569
##########
sql/src/test/java/org/apache/druid/sql/calcite/expression/ExpressionsTest.java:
##########
@@ -1779,8 +1778,7 @@ public void testTimeMinusYearMonthInterval()
DruidExpression.functionCall("timestamp_shift"),
ImmutableList.of(
DruidExpression.ofColumn(ColumnType.LONG, "t"),
- // RexNode type "interval year to month" is not reported as ColumnType.STRING
- DruidExpression.ofLiteral(null, DruidExpression.stringLiteral("P13M")),
+ DruidExpression.ofLiteral(ColumnType.LONG, DruidExpression.stringLiteral("P13M")),
Review Comment:
hmm, so this was a result of merging the rules from cast into `Calcites.getColumnTypeForRelDataType` which had interval types as LONG, but agree it does look a bit strange so will fix to report these interval types as strings instead and see what that breaks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org