You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "raghavyadav01 (via GitHub)" <gi...@apache.org> on 2025/04/21 20:40:49 UTC
[PR] Pass Table Config to timeSeries Plugin to use in logical plan phase. [pinot]
raghavyadav01 opened a new pull request, #15604:
URL: https://github.com/apache/pinot/pull/15604
Commit contains following changes
- Pass Table Configs to TSE query Plugin.
- Table Config will be used build logical plan based on schema.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [timeseries] Add Metadata Provider to Time Series Query Planner [pinot]
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana merged PR #15604:
URL: https://github.com/apache/pinot/pull/15604
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Pass Table Config to timeSeries Plugin to use in logical plan phase. [pinot]
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on code in PR #15604:
URL: https://github.com/apache/pinot/pull/15604#discussion_r2057503988
##########
pinot-plugins/pinot-timeseries-lang/pinot-timeseries-m3ql/src/main/java/org/apache/pinot/tsdb/m3ql/M3TimeSeriesPlanner.java:
##########
@@ -48,7 +49,7 @@ public void init(PinotConfiguration pinotConfiguration) {
}
@Override
- public TimeSeriesLogicalPlanResult plan(RangeTimeSeriesRequest request) {
+ public TimeSeriesLogicalPlanResult plan(RangeTimeSeriesRequest request, List<TableConfig> tableConfigs) {
Review Comment:
I think the right way to do this is to add a Function<String, TableConfig> which can return tableConfigs for a given raw table name.
This is because RangeTimeSeriesRequest is not aware of the tables being queried and that is only known once you parse the query out using language specific parsers.
However, using Function directly is quite limiting and not a great idea, so we can create a new `MetadataProvider` interface which has a single method `List<TableConfig> getTableConfigs(String rawTableName)`. Later we can add more methods to it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Pass Table Config to timeSeries Plugin to use in logical plan phase. [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #15604:
URL: https://github.com/apache/pinot/pull/15604#issuecomment-2819532093
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15604?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `0%` with `6 lines` in your changes missing coverage. Please review.
> Project coverage is 55.69%. Comparing base [(`8e1c206`)](https://app.codecov.io/gh/apache/pinot/commit/8e1c206763677c93f8d6604a3bd0f911242cc146?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`5f5c624`)](https://app.codecov.io/gh/apache/pinot/commit/5f5c624bf04b373e8e521e9b6b953590d313fad9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 2 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/pinot/pull/15604?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...pache/pinot/common/config/provider/TableCache.java](https://app.codecov.io/gh/apache/pinot/pull/15604?src=pr&el=tree&filepath=pinot-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcommon%2Fconfig%2Fprovider%2FTableCache.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3Byb3ZpZGVyL1RhYmxlQ2FjaGUuamF2YQ==) | 0.00% | [5 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/15604?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pinot/tsdb/planner/TimeSeriesQueryEnvironment.java](https://app.codecov.io/gh/apache/pinot/pull/15604?src=pr&el=tree&filepath=pinot-timeseries%2Fpinot-timeseries-planner%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Ftsdb%2Fplanner%2FTimeSeriesQueryEnvironment.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtdGltZXNlcmllcy9waW5vdC10aW1lc2VyaWVzLXBsYW5uZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3RzZGIvcGxhbm5lci9UaW1lU2VyaWVzUXVlcnlFbnZpcm9ubWVudC5qYXZh) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/15604?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8e1c206) and HEAD (5f5c624). Click for more details.
>
> <details><summary>HEAD has 67 uploads less than BASE</summary>
>
>| Flag | BASE (8e1c206) | HEAD (5f5c624) |
>|------|------|------|
>|unittests|6|1|
>|unittests2|3|0|
>|java-21|10|1|
>|skip-bytebuffers-true|4|0|
>|temurin|15|1|
>|unittests1|3|1|
>|skip-bytebuffers-false|8|1|
>|java-11|5|0|
>|integration|9|0|
>|integration2|3|0|
>|integration1|3|0|
>|custom-integration1|3|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #15604 +/- ##
============================================
- Coverage 62.87% 55.69% -7.19%
+ Complexity 1384 682 -702
============================================
Files 2864 2239 -625
Lines 162681 121082 -41599
Branches 24904 19319 -5585
============================================
- Hits 102288 67435 -34853
+ Misses 52696 48101 -4595
+ Partials 7697 5546 -2151
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.69% <0.00%> (-7.12%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.69% <0.00%> (-7.16%)` | :arrow_down: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.69% <0.00%> (-7.19%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.69% <0.00%> (-7.18%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.69% <0.00%> (-0.11%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/15604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/15604?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Pass Table Config to timeSeries Plugin to use in logical plan phase. [pinot]
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on code in PR #15604:
URL: https://github.com/apache/pinot/pull/15604#discussion_r2074099495
##########
pinot-common/src/main/java/org/apache/pinot/common/config/provider/TableCache.java:
##########
@@ -204,6 +204,17 @@ public TableConfig getTableConfig(String tableNameWithType) {
return tableConfigInfo != null ? tableConfigInfo._tableConfig : null;
}
+ /**
+ * Returns all the table configs.
+ */
+ public List<TableConfig> getTableConfigs() {
Review Comment:
This change is not required? I see we are only moving the function around and adding a javadoc
##########
pinot-timeseries/pinot-timeseries-spi/src/main/java/org/apache/pinot/tsdb/spi/TimeSeriesTableMetadata.java:
##########
@@ -0,0 +1,49 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.tsdb.spi;
+
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.data.Schema;
+
+
+/**
+ * Interface for accessing time series table metadata.
+ */
+public interface TimeSeriesTableMetadata {
Review Comment:
nit: can you call it `TimeSeriesMetadata`? In the future this may provide metadata for non-table entities too (e.g. tenant related metadata or instance related metadata)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Pass Table Config to timeSeries Plugin to use in logical plan phase. [pinot]
Posted by "raghavyadav01 (via GitHub)" <gi...@apache.org>.
raghavyadav01 commented on code in PR #15604:
URL: https://github.com/apache/pinot/pull/15604#discussion_r2074023914
##########
pinot-plugins/pinot-timeseries-lang/pinot-timeseries-m3ql/src/main/java/org/apache/pinot/tsdb/m3ql/M3TimeSeriesPlanner.java:
##########
@@ -48,7 +49,7 @@ public void init(PinotConfiguration pinotConfiguration) {
}
@Override
- public TimeSeriesLogicalPlanResult plan(RangeTimeSeriesRequest request) {
+ public TimeSeriesLogicalPlanResult plan(RangeTimeSeriesRequest request, List<TableConfig> tableConfigs) {
Review Comment:
Added a tableMetadata Provider Interface, as of now there are few useful API's in future we can add more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Pass Table Config to timeSeries Plugin to use in logical plan phase. [pinot]
Posted by "raghavyadav01 (via GitHub)" <gi...@apache.org>.
raghavyadav01 commented on code in PR #15604:
URL: https://github.com/apache/pinot/pull/15604#discussion_r2074109868
##########
pinot-common/src/main/java/org/apache/pinot/common/config/provider/TableCache.java:
##########
@@ -204,6 +204,17 @@ public TableConfig getTableConfig(String tableNameWithType) {
return tableConfigInfo != null ? tableConfigInfo._tableConfig : null;
}
+ /**
+ * Returns all the table configs.
+ */
+ public List<TableConfig> getTableConfigs() {
Review Comment:
Yes, Not required anymore. Missed to remove it. Removing it now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [timeseries] Add Metadata Provider to Time Series Query Planner [pinot]
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on code in PR #15604:
URL: https://github.com/apache/pinot/pull/15604#discussion_r2074145355
##########
pinot-timeseries/pinot-timeseries-spi/src/main/java/org/apache/pinot/tsdb/spi/TimeSeriesMetadata.java:
##########
@@ -0,0 +1,49 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.tsdb.spi;
+
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.data.Schema;
+
+
+/**
+ * Interface for accessing time series table metadata.
+ */
+public interface TimeSeriesMetadata {
+ /**
+ * Get the table config for a given table name
+ * @param tableName name of the table
+ * @return table config or null if not found
+ */
+ TableConfig getTableConfig(String tableName);
Review Comment:
Can you add Nullable annotations here too? Ideally Nullable should be in the interface to let users know of potential nulls in the response.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org