You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/10/24 16:20:08 UTC
[GitHub] [pinot] walterddr opened a new pull request, #9643: [multistage] create pinot override SqlOpTable
walterddr opened a new pull request, #9643:
URL: https://github.com/apache/pinot/pull/9643
- use to fix COALESCE parsing issue
- also benefit from future non-function OP registration
- such as standard SQL LIKE but with customization.
- or entirely customized features like #9480
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006261779
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
Review Comment:
not directly from jvm. but yeah this is a good point. given the number of "custom cached lazy getter" we have in the codebase I will do a TODO and clean up later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #9643:
URL: https://github.com/apache/pinot/pull/9643#issuecomment-1289334257
# [Codecov](https://codecov.io/gh/apache/pinot/pull/9643?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#9643](https://codecov.io/gh/apache/pinot/pull/9643?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (89fe91b) into [master](https://codecov.io/gh/apache/pinot/commit/bf70071bc1d488e7b2ad7aeeb832b1e5da230e0c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bf70071) will **decrease** coverage by `2.14%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #9643 +/- ##
============================================
- Coverage 28.01% 25.87% -2.15%
+ Complexity 53 44 -9
============================================
Files 1931 1935 +4
Lines 103532 103816 +284
Branches 15705 15758 +53
============================================
- Hits 29004 26858 -2146
- Misses 71688 74246 +2558
+ Partials 2840 2712 -128
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `25.87% <0.00%> (+0.05%)` | :arrow_up: |
| integration2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/9643?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../java/org/apache/pinot/query/QueryEnvironment.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvUXVlcnlFbnZpcm9ubWVudC5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/common/utils/DateTimeUtils.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0ZVRpbWVVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/server/access/RequesterIdentity.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL2FjY2Vzcy9SZXF1ZXN0ZXJJZGVudGl0eS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...t/core/plan/StreamingInstanceResponsePlanNode.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9wbGFuL1N0cmVhbWluZ0luc3RhbmNlUmVzcG9uc2VQbGFuTm9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ore/operator/streaming/StreamingResponseUtils.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9zdHJlYW1pbmcvU3RyZWFtaW5nUmVzcG9uc2VVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...server/starter/helix/SegmentReloadStatusValue.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9TZWdtZW50UmVsb2FkU3RhdHVzVmFsdWUuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ager/realtime/PeerSchemeSplitSegmentCommitter.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvUGVlclNjaGVtZVNwbGl0U2VnbWVudENvbW1pdHRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...form/function/IsDistinctFromTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSXNEaXN0aW5jdEZyb21UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...m/function/IsNotDistinctFromTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSXNOb3REaXN0aW5jdEZyb21UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...urces/ServerReloadControllerJobStatusResponse.java](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1NlcnZlclJlbG9hZENvbnRyb2xsZXJKb2JTdGF0dXNSZXNwb25zZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [234 more](https://codecov.io/gh/apache/pinot/pull/9643/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on PR #9643:
URL: https://github.com/apache/pinot/pull/9643#issuecomment-1289426336
CC @61yao @agavra ^
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006162207
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
Review Comment:
thanks. just want to make sure this can cover the use cases.
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/QueryEnvironment.java:
##########
@@ -91,7 +91,10 @@ public QueryEnvironment(TypeFactory typeFactory, CalciteSchema rootSchema, Worke
new CalciteConnectionConfigImpl(catalogReaderConfigProperties));
_config = Frameworks.newConfigBuilder().traitDefs()
- .operatorTable(new ChainedSqlOperatorTable(Arrays.asList(SqlStdOperatorTable.instance(), _catalogReader)))
+ .operatorTable(new ChainedSqlOperatorTable(Arrays.asList(
+ PinotOperatorTable.instance(),
+// SqlStdOperatorTable.instance(),
Review Comment:
yeah that was my bad.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006070296
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/QueryEnvironment.java:
##########
@@ -91,7 +91,10 @@ public QueryEnvironment(TypeFactory typeFactory, CalciteSchema rootSchema, Worke
new CalciteConnectionConfigImpl(catalogReaderConfigProperties));
_config = Frameworks.newConfigBuilder().traitDefs()
- .operatorTable(new ChainedSqlOperatorTable(Arrays.asList(SqlStdOperatorTable.instance(), _catalogReader)))
+ .operatorTable(new ChainedSqlOperatorTable(Arrays.asList(
+ PinotOperatorTable.instance(),
+// SqlStdOperatorTable.instance(),
Review Comment:
suggest removing it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006260838
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
+ // Creates and initializes the standard operator table.
+ // Uses two-phase construction, because we can't initialize the
+ // table until the constructor of the sub-class has completed.
+ _instance = new PinotOperatorTable();
+ _instance.initNoDuplicate();
+ }
+ return _instance;
+ }
+
+ /**
+ * Initialize without duplicate, e.g. when 2 duplicate operator is linked with the same op
+ * {@link org.apache.calcite.sql.SqlKind} it causes problem.
+ */
+ public final void initNoDuplicate() {
+ // Use reflection to register the expressions stored in public fields.
+ for (Field field : getClass().getFields()) {
Review Comment:
no we have operators such as LIKE and REGEXP_LIKE and later AT TIME ZONE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006069917
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotSqlCoalesceFunction.java:
##########
@@ -0,0 +1,35 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
Review Comment:
(nit) `sql.func` or `sql.function` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr merged pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr merged PR #9643:
URL: https://github.com/apache/pinot/pull/9643
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006163391
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
+ // Creates and initializes the standard operator table.
+ // Uses two-phase construction, because we can't initialize the
+ // table until the constructor of the sub-class has completed.
+ _instance = new PinotOperatorTable();
+ _instance.initNoDuplicate();
+ }
+ return _instance;
+ }
+
+ /**
+ * Initialize without duplicate, e.g. when 2 duplicate operator is linked with the same op
+ * {@link org.apache.calcite.sql.SqlKind} it causes problem.
+ */
+ public final void initNoDuplicate() {
+ // Use reflection to register the expressions stored in public fields.
+ for (Field field : getClass().getFields()) {
Review Comment:
yeah SqlStdOperatorTable is a reflection-based, i was just copying to code out and modify it.
this is one-time lazy static initialization, so I will leave this in place and add a TODO for perf improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] agavra commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
agavra commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006206151
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
+ // Creates and initializes the standard operator table.
+ // Uses two-phase construction, because we can't initialize the
+ // table until the constructor of the sub-class has completed.
+ _instance = new PinotOperatorTable();
+ _instance.initNoDuplicate();
+ }
+ return _instance;
+ }
+
+ /**
+ * Initialize without duplicate, e.g. when 2 duplicate operator is linked with the same op
+ * {@link org.apache.calcite.sql.SqlKind} it causes problem.
+ */
+ public final void initNoDuplicate() {
+ // Use reflection to register the expressions stored in public fields.
+ for (Field field : getClass().getFields()) {
Review Comment:
sounds good 👍 it was more for clarity (reflection is hard to debug) than for performance but I suspect this is pretty low chance of failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] agavra commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
agavra commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006054905
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
Review Comment:
an alternative, by the way, would be to just register them in a new operator table (like `ListSqlOperatorTable`) here: https://github.com/apache/pinot/blob/46aa54139fcbdab36851dcc2c4349d8ccfe3da79/pinot-query-planner/src/main/java/org/apache/pinot/query/QueryEnvironment.java#L94
not sure what the tradeoffs would be
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006161216
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
Review Comment:
you can't override same SqlKind with chained operator. for example in this case I don't want another "COALESCE", I want calcite to discard the coalesce in the StdOp table.
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
Review Comment:
you can't override same SqlKind with chained operator, only addition. for example in this case I don't want another "COALESCE", I want calcite to discard the coalesce in the StdOp table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006161758
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotSqlCoalesceFunction.java:
##########
@@ -0,0 +1,35 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
Review Comment:
can't change this. it is package in `org.apache.calcite.sql.fun` to access some package private methods
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006260838
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
+ // Creates and initializes the standard operator table.
+ // Uses two-phase construction, because we can't initialize the
+ // table until the constructor of the sub-class has completed.
+ _instance = new PinotOperatorTable();
+ _instance.initNoDuplicate();
+ }
+ return _instance;
+ }
+
+ /**
+ * Initialize without duplicate, e.g. when 2 duplicate operator is linked with the same op
+ * {@link org.apache.calcite.sql.SqlKind} it causes problem.
+ */
+ public final void initNoDuplicate() {
+ // Use reflection to register the expressions stored in public fields.
+ for (Field field : getClass().getFields()) {
Review Comment:
no we have operators such as LIKE and REGEXP_LIKE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] 61yao commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
61yao commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006254457
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
+ // Creates and initializes the standard operator table.
+ // Uses two-phase construction, because we can't initialize the
+ // table until the constructor of the sub-class has completed.
+ _instance = new PinotOperatorTable();
+ _instance.initNoDuplicate();
+ }
+ return _instance;
+ }
+
+ /**
+ * Initialize without duplicate, e.g. when 2 duplicate operator is linked with the same op
+ * {@link org.apache.calcite.sql.SqlKind} it causes problem.
+ */
+ public final void initNoDuplicate() {
+ // Use reflection to register the expressions stored in public fields.
+ for (Field field : getClass().getFields()) {
Review Comment:
Should we just use SqlFunction for now since that's the only thing we have in fields?
Would be good to add a comment to caution on adding fields since this will be used in this func and this needs to be no-dup and it is case-insensitive.
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
Review Comment:
Surprised java doesn't come with initi_once option...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006260838
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
+ // Creates and initializes the standard operator table.
+ // Uses two-phase construction, because we can't initialize the
+ // table until the constructor of the sub-class has completed.
+ _instance = new PinotOperatorTable();
+ _instance.initNoDuplicate();
+ }
+ return _instance;
+ }
+
+ /**
+ * Initialize without duplicate, e.g. when 2 duplicate operator is linked with the same op
+ * {@link org.apache.calcite.sql.SqlKind} it causes problem.
+ */
+ public final void initNoDuplicate() {
+ // Use reflection to register the expressions stored in public fields.
+ for (Field field : getClass().getFields()) {
Review Comment:
no we have operators such as LIKE and REGEXP_LIKE and later AT TIME ZONE
to answer previous questions: no i don't think ops and funcs are treated differently at the moment, the only different is at the lookupOperatorOverloads method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] agavra commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
agavra commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006036963
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
Review Comment:
nit: we can use Guava's `Suppliers#memoize` here if we want which has a safe implementation of DCL all wrapped into a convenient `Supplier` interface
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/QueryEnvironment.java:
##########
@@ -91,7 +91,10 @@ public QueryEnvironment(TypeFactory typeFactory, CalciteSchema rootSchema, Worke
new CalciteConnectionConfigImpl(catalogReaderConfigProperties));
_config = Frameworks.newConfigBuilder().traitDefs()
- .operatorTable(new ChainedSqlOperatorTable(Arrays.asList(SqlStdOperatorTable.instance(), _catalogReader)))
+ .operatorTable(new ChainedSqlOperatorTable(Arrays.asList(
+ PinotOperatorTable.instance(),
+// SqlStdOperatorTable.instance(),
Review Comment:
nit: commented out code
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
+
+ public static synchronized PinotOperatorTable instance() {
+ if (_instance == null) {
+ // Creates and initializes the standard operator table.
+ // Uses two-phase construction, because we can't initialize the
+ // table until the constructor of the sub-class has completed.
+ _instance = new PinotOperatorTable();
+ _instance.initNoDuplicate();
+ }
+ return _instance;
+ }
+
+ /**
+ * Initialize without duplicate, e.g. when 2 duplicate operator is linked with the same op
+ * {@link org.apache.calcite.sql.SqlKind} it causes problem.
+ */
+ public final void initNoDuplicate() {
+ // Use reflection to register the expressions stored in public fields.
+ for (Field field : getClass().getFields()) {
Review Comment:
😬 instead of using reflection can we just have them all in an enum?
```
enum Functions {
COALESCE(new PinotSqlCoalesceFunction());
final SqlOperator _op;
Functions(SqlOperator op) {...}
void register(PinotOperatorTable table) {...}
}
```
and then just call:
```
for (Functions fun : Functions.values()) {
fun.register(this);
}
```
Also is there any reason we differentiate between SqlFunction and SqlOperator? it looks like the code handling them is the same.
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import java.lang.reflect.Field;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.validate.SqlNameMatchers;
+import org.apache.calcite.util.Util;
+import org.checkerframework.checker.nullness.qual.MonotonicNonNull;
+
+
+/**
+ * {@link PinotOperatorTable} defines the {@link SqlOperator} overrides on top of the {@link SqlStdOperatorTable}.
+ *
+ * <p>The main purpose of this Pinot specific SQL operator table is to
+ * <ul>
+ * <li>Ensure that any specific SQL validation rules can apply with Pinot override entirely over Calcite's.</li>
+ * <li>Ability to create customer operators that are not function and cannot use
+ * {@link org.apache.calcite.prepare.Prepare.CatalogReader} to override</li>
+ * <li>Still maintain minimum customization and benefit from Calcite's original operator table setting.</li>
+ * </ul>
+ */
+public class PinotOperatorTable extends SqlStdOperatorTable {
+
+ private static @MonotonicNonNull PinotOperatorTable _instance;
+
+ public static final SqlFunction COALESCE = new PinotSqlCoalesceFunction();
Review Comment:
nice, I think this is cleaner than my approach in https://github.com/apache/pinot/pull/9480 and would work for `AT TIME ZONE` as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9643: [multistage] create pinot override SqlOpTable
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9643:
URL: https://github.com/apache/pinot/pull/9643#discussion_r1006161758
##########
pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotSqlCoalesceFunction.java:
##########
@@ -0,0 +1,35 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.calcite.sql.fun;
Review Comment:
can't change this it is package in `org.apache.calcite` to access some package private methods
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org