You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/01/17 22:33:46 UTC

[GitHub] [incubator-pinot] fx19880617 opened a new pull request #6451: Adding task scheduler for pinot tasks in cron expression

fx19880617 opened a new pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451


   ## Description
   Adding support in task config to let pinot controller schedule the job based on cron expression.
   Sample config:
   ```
   {
       "tableName": "githubEvents",
       "tableType": "OFFLINE",
       "segmentsConfig": {
           "segmentPushType": "APPEND",
           "segmentAssignmentStrategy": "BalanceNumSegmentAssignmentStrategy",
           "schemaName": "githubEvents",
           "replication": "1",
           "timeColumnName": "event_time",
           "timeType": "MILLISECONDS"
       },
       "tenants": {},
       "tableIndexConfig": {
           "starTreeIndexConfigs": [
               {
                   "dimensionsSplitOrder": [
                       "type",
                       "repo_id"
                   ],
                   "skipStarNodeCreationForDimensions": [],
                   "functionColumnPairs": [
                       "SUM__pull_request_additions",
                       "SUM__pull_request_deletions",
                       "SUM__pull_request_changed_files",
                       "COUNT__star",
                       "DISTINCT_COUNT_HLL__actor_id"
                   ],
                   "maxLeafRecords": 1000
               }
           ],
           "enableDynamicStarTreeCreation": true,
           "loadMode": "MMAP",
           "invertedIndexColumns": [],
           "segmentPartitionConfig": {
               "columnPartitionMap": {
                   "repo_id": {
                       "functionName": "Murmur",
                       "numPartitions": 1024
                   }
               }
           },
           "noDictionaryColumns": []
       },
       "routing": {
           "segmentPrunerTypes": [
               "partition"
           ]
       },
       "metadata": {
           "customConfigs": {}
       },
       "ingestionConfig": {
           "batchIngestionConfig": {
               "segmentIngestionType": "APPEND",
               "segmentIngestionFrequency": "DAILY",
               "batchConfigMaps":[
               {
                 "inputDirURI": "s3://my-data/github_flatten/partitioned_by_date/yyyy=2016/mm=1/dd=1/",
                 "inputFormat": "parquet",
                 "includeFileNamePattern": "glob:**/*.parquet",
                 "input.fs.className": "org.apache.pinot.plugin.filesystem.S3PinotFS",
                 "input.fs.prop.region": "us-west-2"
               }],
               "segmentNameSpec": {},
               "pushSpec": {}
           },
           "transformConfigs": [
               {
                   "columnName": "event_time",
                   "transformFunction": "fromDateTime(created_at, \"yyyy-MM-dd'T'HH:mm:ssZ\")"
               }
           ]
       },
       "task": {
           "taskTypeConfigsMap": {
               "SegmentGenerationAndPushTask": {
                 "schedule" : "0 * * ? * * *"
           }
         }
       }
   }
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

codecov-io edited a comment on pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#issuecomment-761898819


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=h1) Report
   > Merging [#6451](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=desc) (e2d607e) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.40%`.
   > The diff coverage is `56.80%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6451/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6451      +/-   ##
   ==========================================
   - Coverage   66.44%   65.04%   -1.41%     
   ==========================================
     Files        1075     1332     +257     
     Lines       54773    65187   +10414     
     Branches     8168     9498    +1330     
   ==========================================
   + Hits        36396    42398    +6002     
   - Misses      15700    19752    +4052     
   - Partials     2677     3037     +360     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.04% <56.80%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [...common/config/tuner/NoOpTableTableConfigTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL05vT3BUYWJsZVRhYmxlQ29uZmlnVHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ot/common/config/tuner/RealTimeAutoIndexTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL1JlYWxUaW1lQXV0b0luZGV4VHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | ... and [1173 more](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=footer). Last update [28882ba...e2d607e](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] fx19880617 merged pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

fx19880617 merged pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io commented on pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

codecov-io commented on pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#issuecomment-761898819


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=h1) Report
   > Merging [#6451](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=desc) (fa192a8) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.37%`.
   > The diff coverage is `56.80%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6451/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6451      +/-   ##
   ==========================================
   - Coverage   66.44%   65.07%   -1.38%     
   ==========================================
     Files        1075     1332     +257     
     Lines       54773    65162   +10389     
     Branches     8168     9502    +1334     
   ==========================================
   + Hits        36396    42406    +6010     
   - Misses      15700    19720    +4020     
   - Partials     2677     3036     +359     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.07% <56.80%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [...common/config/tuner/NoOpTableTableConfigTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL05vT3BUYWJsZVRhYmxlQ29uZmlnVHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ot/common/config/tuner/RealTimeAutoIndexTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL1JlYWxUaW1lQXV0b0luZGV4VHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | ... and [1167 more](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=footer). Last update [832ece7...fa192a8](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

fx19880617 commented on a change in pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#discussion_r559917359



##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java
##########
@@ -62,6 +89,220 @@ public PinotTaskManager(PinotHelixTaskResourceManager helixTaskResourceManager,
     _helixTaskResourceManager = helixTaskResourceManager;
     _clusterInfoAccessor = new ClusterInfoAccessor(helixResourceManager, helixTaskResourceManager, controllerConf);
     _taskGeneratorRegistry = new TaskGeneratorRegistry(_clusterInfoAccessor);
+    try {
+      _scheduledExecutorService = new StdSchedulerFactory().getScheduler();
+      _scheduledExecutorService.start();
+      LOGGER.info("Subscribe to tables change under PropertyStore path: {}", TABLE_CONFIG_PARENT_PATH);
+      _pinotHelixResourceManager.getPropertyStore()
+          .subscribeChildChanges(TABLE_CONFIG_PARENT_PATH, (parentPath, currentChilds) -> {
+            Set<String> tableToAdd = new HashSet(currentChilds);
+            tableToAdd.removeAll(_tableTaskSchedulerUpdaterMap.keySet());
+            for (String tableWithType : tableToAdd) {
+              subscribeTableConfigChanges(tableWithType);
+            }
+            Set<String> tableToDelete = new HashSet(_tableTaskSchedulerUpdaterMap.keySet());
+            tableToDelete.removeAll(currentChilds);
+            if (!tableToDelete.isEmpty()) {
+              LOGGER.info("Found tables to clean up cron task scheduler: {}", tableToDelete);
+              for (String tableWithType : tableToDelete) {
+                cleanUpCronTaskSchedulerForTable(tableWithType);
+              }
+            }
+          });
+      for (String tableWithType : helixResourceManager.getAllTables()) {
+        subscribeTableConfigChanges(tableWithType);
+      }
+    } catch (SchedulerException e) {
+      LOGGER.error("Unable to create a scheduler.", e);
+      _scheduledExecutorService = null;
+    }
+  }
+
+  private String getPropertyStorePathForTable(String tableWithType) {
+    return TABLE_CONFIG_PATH_PREFIX + tableWithType;
+  }
+
+  protected synchronized void cleanupCronTaskScheduler() {
+    try {
+      _scheduledExecutorService.clear();
+    } catch (SchedulerException e) {
+      LOGGER.error("Failed to clear all tasks in scheduler", e);
+    }
+  }
+
+  public synchronized void cleanUpCronTaskSchedulerForTable(String tableWithType) {
+    LOGGER.info("Cleaning up task in scheduler for table {}", tableWithType);
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = _tableTaskSchedulerUpdaterMap.get(tableWithType);
+    _pinotHelixResourceManager.getPropertyStore()
+        .unsubscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    removeAllTasksFromCronExpressions(tableWithType);
+    _tableTaskSchedulerUpdaterMap.remove(tableWithType);
+  }
+
+  private synchronized void removeAllTasksFromCronExpressions(String tableWithType) {
+    Set<JobKey> jobKeys;
+    try {
+      jobKeys = _scheduledExecutorService.getJobKeys(GroupMatcher.anyJobGroup());
+    } catch (SchedulerException e) {
+      LOGGER.error("Got exception when fetching all jobKeys", e);
+      return;
+    }
+    for (JobKey jobKey : jobKeys) {
+      if (jobKey.getName().equalsIgnoreCase(tableWithType)) {
+        try {
+          _scheduledExecutorService.deleteJob(jobKey);
+        } catch (SchedulerException e) {
+          LOGGER.error("Got exception when deleting the scheduled job - {}", jobKey, e);
+        }
+      }
+    }
+    _tableTaskTypeToCronExpressionMap.remove(tableWithType);
+  }
+
+  public synchronized void subscribeTableConfigChanges(String tableWithType) {
+    if (_tableTaskSchedulerUpdaterMap.containsKey(tableWithType)) {
+      return;
+    }
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = new TableTaskSchedulerUpdater(tableWithType, this);
+    _pinotHelixResourceManager.getPropertyStore()
+        .subscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    _tableTaskSchedulerUpdaterMap.put(tableWithType, tableTaskSchedulerUpdater);
+    LOGGER.info("Trying to update task schedule for table: {}", tableWithType);
+    try {
+      updateCronTaskScheduler(tableWithType);

Review comment:
       From the test, this is not expected behavior. I need to call this explicitly for the first time.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

codecov-io edited a comment on pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#issuecomment-762195257


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=h1) Report
   > Merging [#6451](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=desc) (6a0d78b) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `22.74%`.
   > The diff coverage is `38.51%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6451/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #6451       +/-   ##
   ===========================================
   - Coverage   66.44%   43.70%   -22.75%     
   ===========================================
     Files        1075     1332      +257     
     Lines       54773    65236    +10463     
     Branches     8168     9506     +1338     
   ===========================================
   - Hits        36396    28510     -7886     
   - Misses      15700    34341    +18641     
   + Partials     2677     2385      -292     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `43.70% <38.51%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...org/apache/pinot/broker/queryquota/HitCounter.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9IaXRDb3VudGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...che/pinot/broker/queryquota/MaxHitRateTracker.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9NYXhIaXRSYXRlVHJhY2tlci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ache/pinot/broker/queryquota/QueryQuotaEntity.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9RdWVyeVF1b3RhRW50aXR5LmphdmE=) | `0.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ceselector/StrictReplicaGroupInstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL1N0cmljdFJlcGxpY2FHcm91cEluc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...roker/routing/segmentpruner/TimeSegmentPruner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL1RpbWVTZWdtZW50UHJ1bmVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...roker/routing/segmentpruner/interval/Interval.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL2ludGVydmFsL0ludGVydmFsLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...r/routing/segmentpruner/interval/IntervalTree.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL2ludGVydmFsL0ludGVydmFsVHJlZS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | ... and [1321 more](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=footer). Last update [28882ba...6a0d78b](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

Jackie-Jiang commented on a change in pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#discussion_r559747469



##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java
##########
@@ -503,6 +508,21 @@ private void verifyTableConfigs(TableConfig newTableConfig) {
               existingTimeColumnType, newTimeColumnType));
     }
 
+    // Check if task schedule is valid.
+    TableTaskConfig taskConfig = newTableConfig.getTaskConfig();
+    if (taskConfig != null && taskConfig.isTaskTypeEnabled(MinionConstants.SegmentGenerationAndPushTask.TASK_TYPE)) {
+      Map<String, String> taskTypeConfig =
+          taskConfig.getConfigsForTaskType(MinionConstants.SegmentGenerationAndPushTask.TASK_TYPE);
+      if (taskTypeConfig != null && taskTypeConfig.containsKey(PinotTaskManager.SCHEDULE_KEY)) {
+        String cronExprStr = taskTypeConfig.get(PinotTaskManager.SCHEDULE_KEY);
+        try {
+          CronScheduleBuilder.cronSchedule(cronExprStr);
+        } catch (Exception e) {
+          throw new PinotHelixResourceManager.InvalidTableConfigException(
+              String.format("Task type contains an invalid cron schedule: %s", cronExprStr), e);

Review comment:
       ```suggestion
                 String.format("SegmentGenerationAndPushTask contains an invalid cron schedule: %s", cronExprStr), e);
   ```

##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java
##########
@@ -62,6 +89,220 @@ public PinotTaskManager(PinotHelixTaskResourceManager helixTaskResourceManager,
     _helixTaskResourceManager = helixTaskResourceManager;
     _clusterInfoAccessor = new ClusterInfoAccessor(helixResourceManager, helixTaskResourceManager, controllerConf);
     _taskGeneratorRegistry = new TaskGeneratorRegistry(_clusterInfoAccessor);
+    try {
+      _scheduledExecutorService = new StdSchedulerFactory().getScheduler();
+      _scheduledExecutorService.start();
+      LOGGER.info("Subscribe to tables change under PropertyStore path: {}", TABLE_CONFIG_PARENT_PATH);
+      _pinotHelixResourceManager.getPropertyStore()
+          .subscribeChildChanges(TABLE_CONFIG_PARENT_PATH, (parentPath, currentChilds) -> {
+            Set<String> tableToAdd = new HashSet(currentChilds);
+            tableToAdd.removeAll(_tableTaskSchedulerUpdaterMap.keySet());
+            for (String tableWithType : tableToAdd) {
+              subscribeTableConfigChanges(tableWithType);
+            }
+            Set<String> tableToDelete = new HashSet(_tableTaskSchedulerUpdaterMap.keySet());
+            tableToDelete.removeAll(currentChilds);
+            if (!tableToDelete.isEmpty()) {
+              LOGGER.info("Found tables to clean up cron task scheduler: {}", tableToDelete);
+              for (String tableWithType : tableToDelete) {
+                cleanUpCronTaskSchedulerForTable(tableWithType);
+              }
+            }
+          });
+      for (String tableWithType : helixResourceManager.getAllTables()) {
+        subscribeTableConfigChanges(tableWithType);
+      }
+    } catch (SchedulerException e) {
+      LOGGER.error("Unable to create a scheduler.", e);
+      _scheduledExecutorService = null;
+    }
+  }
+
+  private String getPropertyStorePathForTable(String tableWithType) {
+    return TABLE_CONFIG_PATH_PREFIX + tableWithType;
+  }
+
+  protected synchronized void cleanupCronTaskScheduler() {
+    try {
+      _scheduledExecutorService.clear();
+    } catch (SchedulerException e) {
+      LOGGER.error("Failed to clear all tasks in scheduler", e);
+    }
+  }
+
+  public synchronized void cleanUpCronTaskSchedulerForTable(String tableWithType) {
+    LOGGER.info("Cleaning up task in scheduler for table {}", tableWithType);
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = _tableTaskSchedulerUpdaterMap.get(tableWithType);
+    _pinotHelixResourceManager.getPropertyStore()
+        .unsubscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    removeAllTasksFromCronExpressions(tableWithType);
+    _tableTaskSchedulerUpdaterMap.remove(tableWithType);
+  }
+
+  private synchronized void removeAllTasksFromCronExpressions(String tableWithType) {
+    Set<JobKey> jobKeys;
+    try {
+      jobKeys = _scheduledExecutorService.getJobKeys(GroupMatcher.anyJobGroup());
+    } catch (SchedulerException e) {
+      LOGGER.error("Got exception when fetching all jobKeys", e);
+      return;
+    }
+    for (JobKey jobKey : jobKeys) {
+      if (jobKey.getName().equalsIgnoreCase(tableWithType)) {
+        try {
+          _scheduledExecutorService.deleteJob(jobKey);
+        } catch (SchedulerException e) {
+          LOGGER.error("Got exception when deleting the scheduled job - {}", jobKey, e);
+        }
+      }
+    }
+    _tableTaskTypeToCronExpressionMap.remove(tableWithType);
+  }
+
+  public synchronized void subscribeTableConfigChanges(String tableWithType) {
+    if (_tableTaskSchedulerUpdaterMap.containsKey(tableWithType)) {
+      return;
+    }
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = new TableTaskSchedulerUpdater(tableWithType, this);
+    _pinotHelixResourceManager.getPropertyStore()
+        .subscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    _tableTaskSchedulerUpdaterMap.put(tableWithType, tableTaskSchedulerUpdater);
+    LOGGER.info("Trying to update task schedule for table: {}", tableWithType);
+    try {
+      updateCronTaskScheduler(tableWithType);
+    } catch (Exception e) {
+      LOGGER.error("Got exception during updateCronTaskScheduler for {}", tableWithType, e);
+    }
+  }
+
+  public synchronized void updateCronTaskScheduler(String tableWithType) {
+    if (_scheduledExecutorService == null) {
+      return;
+    }
+    TableConfig tableConfig = _pinotHelixResourceManager.getTableConfig(tableWithType);
+    TableTaskConfig taskConfig = tableConfig.getTaskConfig();

Review comment:
       Do a `null` check first in case table is removed?

##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java
##########
@@ -62,6 +89,220 @@ public PinotTaskManager(PinotHelixTaskResourceManager helixTaskResourceManager,
     _helixTaskResourceManager = helixTaskResourceManager;
     _clusterInfoAccessor = new ClusterInfoAccessor(helixResourceManager, helixTaskResourceManager, controllerConf);
     _taskGeneratorRegistry = new TaskGeneratorRegistry(_clusterInfoAccessor);
+    try {
+      _scheduledExecutorService = new StdSchedulerFactory().getScheduler();
+      _scheduledExecutorService.start();
+      LOGGER.info("Subscribe to tables change under PropertyStore path: {}", TABLE_CONFIG_PARENT_PATH);
+      _pinotHelixResourceManager.getPropertyStore()
+          .subscribeChildChanges(TABLE_CONFIG_PARENT_PATH, (parentPath, currentChilds) -> {
+            Set<String> tableToAdd = new HashSet(currentChilds);
+            tableToAdd.removeAll(_tableTaskSchedulerUpdaterMap.keySet());
+            for (String tableWithType : tableToAdd) {
+              subscribeTableConfigChanges(tableWithType);
+            }
+            Set<String> tableToDelete = new HashSet(_tableTaskSchedulerUpdaterMap.keySet());
+            tableToDelete.removeAll(currentChilds);
+            if (!tableToDelete.isEmpty()) {
+              LOGGER.info("Found tables to clean up cron task scheduler: {}", tableToDelete);
+              for (String tableWithType : tableToDelete) {
+                cleanUpCronTaskSchedulerForTable(tableWithType);
+              }
+            }
+          });
+      for (String tableWithType : helixResourceManager.getAllTables()) {
+        subscribeTableConfigChanges(tableWithType);
+      }
+    } catch (SchedulerException e) {
+      LOGGER.error("Unable to create a scheduler.", e);
+      _scheduledExecutorService = null;
+    }
+  }
+
+  private String getPropertyStorePathForTable(String tableWithType) {
+    return TABLE_CONFIG_PATH_PREFIX + tableWithType;
+  }
+
+  protected synchronized void cleanupCronTaskScheduler() {
+    try {
+      _scheduledExecutorService.clear();
+    } catch (SchedulerException e) {
+      LOGGER.error("Failed to clear all tasks in scheduler", e);
+    }
+  }
+
+  public synchronized void cleanUpCronTaskSchedulerForTable(String tableWithType) {
+    LOGGER.info("Cleaning up task in scheduler for table {}", tableWithType);
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = _tableTaskSchedulerUpdaterMap.get(tableWithType);
+    _pinotHelixResourceManager.getPropertyStore()
+        .unsubscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    removeAllTasksFromCronExpressions(tableWithType);
+    _tableTaskSchedulerUpdaterMap.remove(tableWithType);
+  }
+
+  private synchronized void removeAllTasksFromCronExpressions(String tableWithType) {
+    Set<JobKey> jobKeys;
+    try {
+      jobKeys = _scheduledExecutorService.getJobKeys(GroupMatcher.anyJobGroup());
+    } catch (SchedulerException e) {
+      LOGGER.error("Got exception when fetching all jobKeys", e);
+      return;
+    }
+    for (JobKey jobKey : jobKeys) {
+      if (jobKey.getName().equalsIgnoreCase(tableWithType)) {
+        try {
+          _scheduledExecutorService.deleteJob(jobKey);
+        } catch (SchedulerException e) {
+          LOGGER.error("Got exception when deleting the scheduled job - {}", jobKey, e);
+        }
+      }
+    }
+    _tableTaskTypeToCronExpressionMap.remove(tableWithType);
+  }
+
+  public synchronized void subscribeTableConfigChanges(String tableWithType) {
+    if (_tableTaskSchedulerUpdaterMap.containsKey(tableWithType)) {
+      return;
+    }
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = new TableTaskSchedulerUpdater(tableWithType, this);
+    _pinotHelixResourceManager.getPropertyStore()
+        .subscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    _tableTaskSchedulerUpdaterMap.put(tableWithType, tableTaskSchedulerUpdater);
+    LOGGER.info("Trying to update task schedule for table: {}", tableWithType);
+    try {
+      updateCronTaskScheduler(tableWithType);

Review comment:
       This is redundant. The callback should be triggered when subscribing the path

##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java
##########
@@ -62,6 +89,220 @@ public PinotTaskManager(PinotHelixTaskResourceManager helixTaskResourceManager,
     _helixTaskResourceManager = helixTaskResourceManager;
     _clusterInfoAccessor = new ClusterInfoAccessor(helixResourceManager, helixTaskResourceManager, controllerConf);
     _taskGeneratorRegistry = new TaskGeneratorRegistry(_clusterInfoAccessor);
+    try {
+      _scheduledExecutorService = new StdSchedulerFactory().getScheduler();
+      _scheduledExecutorService.start();
+      LOGGER.info("Subscribe to tables change under PropertyStore path: {}", TABLE_CONFIG_PARENT_PATH);
+      _pinotHelixResourceManager.getPropertyStore()
+          .subscribeChildChanges(TABLE_CONFIG_PARENT_PATH, (parentPath, currentChilds) -> {
+            Set<String> tableToAdd = new HashSet(currentChilds);
+            tableToAdd.removeAll(_tableTaskSchedulerUpdaterMap.keySet());
+            for (String tableWithType : tableToAdd) {
+              subscribeTableConfigChanges(tableWithType);
+            }
+            Set<String> tableToDelete = new HashSet(_tableTaskSchedulerUpdaterMap.keySet());
+            tableToDelete.removeAll(currentChilds);
+            if (!tableToDelete.isEmpty()) {
+              LOGGER.info("Found tables to clean up cron task scheduler: {}", tableToDelete);
+              for (String tableWithType : tableToDelete) {
+                cleanUpCronTaskSchedulerForTable(tableWithType);
+              }
+            }
+          });
+      for (String tableWithType : helixResourceManager.getAllTables()) {
+        subscribeTableConfigChanges(tableWithType);
+      }
+    } catch (SchedulerException e) {
+      LOGGER.error("Unable to create a scheduler.", e);
+      _scheduledExecutorService = null;
+    }
+  }
+
+  private String getPropertyStorePathForTable(String tableWithType) {
+    return TABLE_CONFIG_PATH_PREFIX + tableWithType;
+  }
+
+  protected synchronized void cleanupCronTaskScheduler() {
+    try {
+      _scheduledExecutorService.clear();
+    } catch (SchedulerException e) {
+      LOGGER.error("Failed to clear all tasks in scheduler", e);
+    }
+  }
+
+  public synchronized void cleanUpCronTaskSchedulerForTable(String tableWithType) {
+    LOGGER.info("Cleaning up task in scheduler for table {}", tableWithType);
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = _tableTaskSchedulerUpdaterMap.get(tableWithType);
+    _pinotHelixResourceManager.getPropertyStore()
+        .unsubscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    removeAllTasksFromCronExpressions(tableWithType);
+    _tableTaskSchedulerUpdaterMap.remove(tableWithType);
+  }
+
+  private synchronized void removeAllTasksFromCronExpressions(String tableWithType) {
+    Set<JobKey> jobKeys;
+    try {
+      jobKeys = _scheduledExecutorService.getJobKeys(GroupMatcher.anyJobGroup());
+    } catch (SchedulerException e) {
+      LOGGER.error("Got exception when fetching all jobKeys", e);
+      return;
+    }
+    for (JobKey jobKey : jobKeys) {
+      if (jobKey.getName().equalsIgnoreCase(tableWithType)) {
+        try {
+          _scheduledExecutorService.deleteJob(jobKey);
+        } catch (SchedulerException e) {
+          LOGGER.error("Got exception when deleting the scheduled job - {}", jobKey, e);
+        }
+      }
+    }
+    _tableTaskTypeToCronExpressionMap.remove(tableWithType);
+  }
+
+  public synchronized void subscribeTableConfigChanges(String tableWithType) {
+    if (_tableTaskSchedulerUpdaterMap.containsKey(tableWithType)) {
+      return;
+    }
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = new TableTaskSchedulerUpdater(tableWithType, this);
+    _pinotHelixResourceManager.getPropertyStore()
+        .subscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    _tableTaskSchedulerUpdaterMap.put(tableWithType, tableTaskSchedulerUpdater);
+    LOGGER.info("Trying to update task schedule for table: {}", tableWithType);
+    try {
+      updateCronTaskScheduler(tableWithType);
+    } catch (Exception e) {
+      LOGGER.error("Got exception during updateCronTaskScheduler for {}", tableWithType, e);
+    }
+  }
+
+  public synchronized void updateCronTaskScheduler(String tableWithType) {
+    if (_scheduledExecutorService == null) {
+      return;
+    }
+    TableConfig tableConfig = _pinotHelixResourceManager.getTableConfig(tableWithType);
+    TableTaskConfig taskConfig = tableConfig.getTaskConfig();
+    if (taskConfig == null) {
+      LOGGER.info("taskConfig is null, trying to remove all the tasks for table {} if any", tableWithType);
+      removeAllTasksFromCronExpressions(tableWithType);
+      return;
+    }
+    Map<String, Map<String, String>> taskTypeConfigsMap = taskConfig.getTaskTypeConfigsMap();
+    if (taskTypeConfigsMap == null) {
+      LOGGER.info("taskTypeConfigsMap is null, trying to remove all the tasks for table {} if any", tableWithType);
+      removeAllTasksFromCronExpressions(tableWithType);
+      return;
+    }
+    Map<String, String> taskToCronExpressionMap = getTaskToCronExpressionMap(taskTypeConfigsMap);
+    LOGGER.info("Got taskToCronExpressionMap {} ", taskToCronExpressionMap);
+    updateCronTaskScheduler(tableWithType, taskToCronExpressionMap);
+  }
+
+  private void updateCronTaskScheduler(String tableWithType, Map<String, String> taskToCronExpressionMap) {
+    if (_scheduledExecutorService == null) {
+      return;
+    }
+    if (_tableTaskTypeToCronExpressionMap.containsKey(tableWithType)) {
+      Map<String, String> existingScheduledTasks = _tableTaskTypeToCronExpressionMap.get(tableWithType);
+      for (String existingTaskType : existingScheduledTasks.keySet()) {
+        // Task should be removed
+        if (!taskToCronExpressionMap.containsKey(existingTaskType)) {
+          try {
+            _scheduledExecutorService.deleteJob(JobKey.jobKey(tableWithType, existingTaskType));
+          } catch (SchedulerException e) {
+            LOGGER.error("Failed to delete scheduled job for table {}, task type {}", tableWithType,
+                existingScheduledTasks, e);
+          }
+          continue;
+        }
+        String existingCronExpression = existingScheduledTasks.get(existingTaskType);
+        String newCronExpression = taskToCronExpressionMap.get(existingTaskType);
+        // Schedule new job
+        if (existingCronExpression == null) {
+          try {
+            scheduleJob(tableWithType, existingTaskType, newCronExpression);
+          } catch (SchedulerException e) {
+            LOGGER.error("Failed to schedule cron task for table {}, task {}, cron expr {}", tableWithType,
+                existingTaskType, newCronExpression, e);
+          }
+          continue;
+        }
+        // Update existing task with new cron expr
+        if (!existingCronExpression.equalsIgnoreCase(newCronExpression)) {
+          try {
+            TriggerKey triggerKey = TriggerKey.triggerKey(tableWithType, existingTaskType);
+            Trigger trigger = TriggerBuilder.newTrigger().withIdentity(triggerKey)
+                .withSchedule(CronScheduleBuilder.cronSchedule(newCronExpression)).build();
+            _scheduledExecutorService.rescheduleJob(triggerKey, trigger);
+          } catch (SchedulerException e) {
+            LOGGER.error("Failed to delete scheduled job for table {}, task type {}", tableWithType,
+                existingScheduledTasks, e);
+          }
+          continue;

Review comment:
       (nit) redundant

##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java
##########
@@ -62,6 +89,220 @@ public PinotTaskManager(PinotHelixTaskResourceManager helixTaskResourceManager,
     _helixTaskResourceManager = helixTaskResourceManager;
     _clusterInfoAccessor = new ClusterInfoAccessor(helixResourceManager, helixTaskResourceManager, controllerConf);
     _taskGeneratorRegistry = new TaskGeneratorRegistry(_clusterInfoAccessor);
+    try {
+      _scheduledExecutorService = new StdSchedulerFactory().getScheduler();
+      _scheduledExecutorService.start();
+      LOGGER.info("Subscribe to tables change under PropertyStore path: {}", TABLE_CONFIG_PARENT_PATH);
+      _pinotHelixResourceManager.getPropertyStore()
+          .subscribeChildChanges(TABLE_CONFIG_PARENT_PATH, (parentPath, currentChilds) -> {
+            Set<String> tableToAdd = new HashSet(currentChilds);
+            tableToAdd.removeAll(_tableTaskSchedulerUpdaterMap.keySet());
+            for (String tableWithType : tableToAdd) {
+              subscribeTableConfigChanges(tableWithType);
+            }
+            Set<String> tableToDelete = new HashSet(_tableTaskSchedulerUpdaterMap.keySet());
+            tableToDelete.removeAll(currentChilds);
+            if (!tableToDelete.isEmpty()) {
+              LOGGER.info("Found tables to clean up cron task scheduler: {}", tableToDelete);
+              for (String tableWithType : tableToDelete) {
+                cleanUpCronTaskSchedulerForTable(tableWithType);
+              }
+            }
+          });
+      for (String tableWithType : helixResourceManager.getAllTables()) {
+        subscribeTableConfigChanges(tableWithType);
+      }
+    } catch (SchedulerException e) {
+      LOGGER.error("Unable to create a scheduler.", e);
+      _scheduledExecutorService = null;
+    }
+  }
+
+  private String getPropertyStorePathForTable(String tableWithType) {
+    return TABLE_CONFIG_PATH_PREFIX + tableWithType;
+  }
+
+  protected synchronized void cleanupCronTaskScheduler() {
+    try {
+      _scheduledExecutorService.clear();
+    } catch (SchedulerException e) {
+      LOGGER.error("Failed to clear all tasks in scheduler", e);
+    }
+  }
+
+  public synchronized void cleanUpCronTaskSchedulerForTable(String tableWithType) {
+    LOGGER.info("Cleaning up task in scheduler for table {}", tableWithType);
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = _tableTaskSchedulerUpdaterMap.get(tableWithType);
+    _pinotHelixResourceManager.getPropertyStore()

Review comment:
       Check if `tableTaskSchedulerUpdater` is `null`. This method might be called multiple times from the callback functions

##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java
##########
@@ -62,6 +89,220 @@ public PinotTaskManager(PinotHelixTaskResourceManager helixTaskResourceManager,
     _helixTaskResourceManager = helixTaskResourceManager;
     _clusterInfoAccessor = new ClusterInfoAccessor(helixResourceManager, helixTaskResourceManager, controllerConf);
     _taskGeneratorRegistry = new TaskGeneratorRegistry(_clusterInfoAccessor);
+    try {
+      _scheduledExecutorService = new StdSchedulerFactory().getScheduler();
+      _scheduledExecutorService.start();
+      LOGGER.info("Subscribe to tables change under PropertyStore path: {}", TABLE_CONFIG_PARENT_PATH);
+      _pinotHelixResourceManager.getPropertyStore()
+          .subscribeChildChanges(TABLE_CONFIG_PARENT_PATH, (parentPath, currentChilds) -> {
+            Set<String> tableToAdd = new HashSet(currentChilds);
+            tableToAdd.removeAll(_tableTaskSchedulerUpdaterMap.keySet());
+            for (String tableWithType : tableToAdd) {
+              subscribeTableConfigChanges(tableWithType);
+            }
+            Set<String> tableToDelete = new HashSet(_tableTaskSchedulerUpdaterMap.keySet());
+            tableToDelete.removeAll(currentChilds);
+            if (!tableToDelete.isEmpty()) {
+              LOGGER.info("Found tables to clean up cron task scheduler: {}", tableToDelete);
+              for (String tableWithType : tableToDelete) {
+                cleanUpCronTaskSchedulerForTable(tableWithType);
+              }
+            }
+          });
+      for (String tableWithType : helixResourceManager.getAllTables()) {
+        subscribeTableConfigChanges(tableWithType);
+      }
+    } catch (SchedulerException e) {
+      LOGGER.error("Unable to create a scheduler.", e);
+      _scheduledExecutorService = null;
+    }
+  }
+
+  private String getPropertyStorePathForTable(String tableWithType) {
+    return TABLE_CONFIG_PATH_PREFIX + tableWithType;
+  }
+
+  protected synchronized void cleanupCronTaskScheduler() {
+    try {
+      _scheduledExecutorService.clear();
+    } catch (SchedulerException e) {
+      LOGGER.error("Failed to clear all tasks in scheduler", e);
+    }
+  }
+
+  public synchronized void cleanUpCronTaskSchedulerForTable(String tableWithType) {
+    LOGGER.info("Cleaning up task in scheduler for table {}", tableWithType);
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = _tableTaskSchedulerUpdaterMap.get(tableWithType);
+    _pinotHelixResourceManager.getPropertyStore()
+        .unsubscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    removeAllTasksFromCronExpressions(tableWithType);
+    _tableTaskSchedulerUpdaterMap.remove(tableWithType);
+  }
+
+  private synchronized void removeAllTasksFromCronExpressions(String tableWithType) {
+    Set<JobKey> jobKeys;
+    try {
+      jobKeys = _scheduledExecutorService.getJobKeys(GroupMatcher.anyJobGroup());
+    } catch (SchedulerException e) {
+      LOGGER.error("Got exception when fetching all jobKeys", e);
+      return;
+    }
+    for (JobKey jobKey : jobKeys) {
+      if (jobKey.getName().equalsIgnoreCase(tableWithType)) {
+        try {
+          _scheduledExecutorService.deleteJob(jobKey);
+        } catch (SchedulerException e) {
+          LOGGER.error("Got exception when deleting the scheduled job - {}", jobKey, e);
+        }
+      }
+    }
+    _tableTaskTypeToCronExpressionMap.remove(tableWithType);
+  }
+
+  public synchronized void subscribeTableConfigChanges(String tableWithType) {
+    if (_tableTaskSchedulerUpdaterMap.containsKey(tableWithType)) {
+      return;
+    }
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = new TableTaskSchedulerUpdater(tableWithType, this);
+    _pinotHelixResourceManager.getPropertyStore()
+        .subscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    _tableTaskSchedulerUpdaterMap.put(tableWithType, tableTaskSchedulerUpdater);
+    LOGGER.info("Trying to update task schedule for table: {}", tableWithType);
+    try {
+      updateCronTaskScheduler(tableWithType);
+    } catch (Exception e) {
+      LOGGER.error("Got exception during updateCronTaskScheduler for {}", tableWithType, e);
+    }
+  }
+
+  public synchronized void updateCronTaskScheduler(String tableWithType) {
+    if (_scheduledExecutorService == null) {
+      return;
+    }
+    TableConfig tableConfig = _pinotHelixResourceManager.getTableConfig(tableWithType);
+    TableTaskConfig taskConfig = tableConfig.getTaskConfig();
+    if (taskConfig == null) {
+      LOGGER.info("taskConfig is null, trying to remove all the tasks for table {} if any", tableWithType);
+      removeAllTasksFromCronExpressions(tableWithType);
+      return;
+    }
+    Map<String, Map<String, String>> taskTypeConfigsMap = taskConfig.getTaskTypeConfigsMap();
+    if (taskTypeConfigsMap == null) {
+      LOGGER.info("taskTypeConfigsMap is null, trying to remove all the tasks for table {} if any", tableWithType);
+      removeAllTasksFromCronExpressions(tableWithType);
+      return;
+    }
+    Map<String, String> taskToCronExpressionMap = getTaskToCronExpressionMap(taskTypeConfigsMap);
+    LOGGER.info("Got taskToCronExpressionMap {} ", taskToCronExpressionMap);
+    updateCronTaskScheduler(tableWithType, taskToCronExpressionMap);
+  }
+
+  private void updateCronTaskScheduler(String tableWithType, Map<String, String> taskToCronExpressionMap) {
+    if (_scheduledExecutorService == null) {
+      return;
+    }
+    if (_tableTaskTypeToCronExpressionMap.containsKey(tableWithType)) {

Review comment:
       (nit) use `get(tableWithType)` and `null` check to save one map lookup. Same for other `containsKey()` followed by `get()`

##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java
##########
@@ -62,6 +89,220 @@ public PinotTaskManager(PinotHelixTaskResourceManager helixTaskResourceManager,
     _helixTaskResourceManager = helixTaskResourceManager;
     _clusterInfoAccessor = new ClusterInfoAccessor(helixResourceManager, helixTaskResourceManager, controllerConf);
     _taskGeneratorRegistry = new TaskGeneratorRegistry(_clusterInfoAccessor);
+    try {
+      _scheduledExecutorService = new StdSchedulerFactory().getScheduler();
+      _scheduledExecutorService.start();
+      LOGGER.info("Subscribe to tables change under PropertyStore path: {}", TABLE_CONFIG_PARENT_PATH);
+      _pinotHelixResourceManager.getPropertyStore()
+          .subscribeChildChanges(TABLE_CONFIG_PARENT_PATH, (parentPath, currentChilds) -> {
+            Set<String> tableToAdd = new HashSet(currentChilds);
+            tableToAdd.removeAll(_tableTaskSchedulerUpdaterMap.keySet());
+            for (String tableWithType : tableToAdd) {
+              subscribeTableConfigChanges(tableWithType);
+            }
+            Set<String> tableToDelete = new HashSet(_tableTaskSchedulerUpdaterMap.keySet());
+            tableToDelete.removeAll(currentChilds);
+            if (!tableToDelete.isEmpty()) {
+              LOGGER.info("Found tables to clean up cron task scheduler: {}", tableToDelete);
+              for (String tableWithType : tableToDelete) {
+                cleanUpCronTaskSchedulerForTable(tableWithType);
+              }
+            }
+          });
+      for (String tableWithType : helixResourceManager.getAllTables()) {
+        subscribeTableConfigChanges(tableWithType);
+      }
+    } catch (SchedulerException e) {
+      LOGGER.error("Unable to create a scheduler.", e);
+      _scheduledExecutorService = null;
+    }
+  }
+
+  private String getPropertyStorePathForTable(String tableWithType) {
+    return TABLE_CONFIG_PATH_PREFIX + tableWithType;
+  }
+
+  protected synchronized void cleanupCronTaskScheduler() {
+    try {
+      _scheduledExecutorService.clear();
+    } catch (SchedulerException e) {
+      LOGGER.error("Failed to clear all tasks in scheduler", e);
+    }
+  }
+
+  public synchronized void cleanUpCronTaskSchedulerForTable(String tableWithType) {
+    LOGGER.info("Cleaning up task in scheduler for table {}", tableWithType);
+    TableTaskSchedulerUpdater tableTaskSchedulerUpdater = _tableTaskSchedulerUpdaterMap.get(tableWithType);
+    _pinotHelixResourceManager.getPropertyStore()
+        .unsubscribeDataChanges(getPropertyStorePathForTable(tableWithType), tableTaskSchedulerUpdater);
+    removeAllTasksFromCronExpressions(tableWithType);
+    _tableTaskSchedulerUpdaterMap.remove(tableWithType);
+  }
+
+  private synchronized void removeAllTasksFromCronExpressions(String tableWithType) {
+    Set<JobKey> jobKeys;
+    try {
+      jobKeys = _scheduledExecutorService.getJobKeys(GroupMatcher.anyJobGroup());
+    } catch (SchedulerException e) {
+      LOGGER.error("Got exception when fetching all jobKeys", e);
+      return;
+    }
+    for (JobKey jobKey : jobKeys) {
+      if (jobKey.getName().equalsIgnoreCase(tableWithType)) {

Review comment:
       I feel it should be exact match instead of case-insensitive match?

##########
File path: pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManagerTest.java
##########
@@ -0,0 +1,134 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.controller.helix.core.minion;
+
+import com.google.common.collect.ImmutableMap;
+import java.util.List;
+import java.util.Set;
+import org.apache.pinot.controller.helix.ControllerTest;
+import org.apache.pinot.core.common.MinionConstants;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableTaskConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.quartz.CronTrigger;
+import org.quartz.JobDetail;
+import org.quartz.JobKey;
+import org.quartz.Scheduler;
+import org.quartz.Trigger;
+import org.quartz.impl.matchers.GroupMatcher;
+import org.testng.Assert;
+import org.testng.annotations.AfterClass;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class PinotTaskManagerTest extends ControllerTest {
+
+  private static final String RAW_TABLE_NAME = "myTable";
+
+  @BeforeClass
+  public void setup()
+      throws Exception {
+    startZk();
+    startController();
+    addFakeBrokerInstancesToAutoJoinHelixCluster(1, true);
+    addFakeServerInstancesToAutoJoinHelixCluster(1, true);
+
+    Schema schema = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+        .addSingleValueDimension("myMap", FieldSpec.DataType.STRING)
+        .addSingleValueDimension("myMapStr", FieldSpec.DataType.STRING)
+        .addSingleValueDimension("complexMapStr", FieldSpec.DataType.STRING).build();
+    addSchema(schema);
+  }
+
+  @Test
+  public void testPinotTaskManagerSchedulerWithUpdate()
+      throws Exception {
+    PinotTaskManager taskManager = _controllerStarter.getTaskManager();
+    Scheduler scheduler = taskManager.getScheduler();
+    Assert.assertNotNull(scheduler);
+
+    // 1. Add Table
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).setTaskConfig(
+        new TableTaskConfig(
+            ImmutableMap.of("SegmentGenerationAndPushTask", ImmutableMap.of("schedule", "0 */10 * ? * * *")))).build();
+    addTableConfig(tableConfig);
+    Thread.sleep(2000);
+    List<String> jobGroupNames = scheduler.getJobGroupNames();
+    Assert.assertEquals(jobGroupNames.size(), 1);
+    for (String group : jobGroupNames) {
+      Set<JobKey> jobKeys = scheduler.getJobKeys(GroupMatcher.groupEquals(group));
+      for (JobKey jobKey : jobKeys) {
+        JobDetail jobDetail = scheduler.getJobDetail(jobKey);
+        Assert.assertEquals(jobDetail.getJobClass(), CronJobScheduleJob.class);
+        Assert.assertEquals(jobDetail.getKey().getName(), RAW_TABLE_NAME + "_OFFLINE");
+        Assert.assertEquals(jobDetail.getKey().getGroup(), MinionConstants.SegmentGenerationAndPushTask.TASK_TYPE);
+        Assert.assertEquals(jobDetail.getJobDataMap().get("PinotTaskManager"), taskManager);
+        Assert.assertEquals(jobDetail.getJobDataMap().get("LeadControllerManager"),
+            _controllerStarter.getLeadControllerManager());
+        List<? extends Trigger> triggersOfJob = scheduler.getTriggersOfJob(jobKey);
+        Assert.assertEquals(triggersOfJob.size(), 1);
+        for (Trigger trigger : triggersOfJob) {
+          Assert.assertTrue(trigger instanceof CronTrigger);
+          Assert.assertEquals(((CronTrigger) trigger).getCronExpression(), "0 */10 * ? * * *");
+        }
+      }
+    }
+
+    // 2. Update table to new schedule
+    tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).setTaskConfig(
+        new TableTaskConfig(
+            ImmutableMap.of("SegmentGenerationAndPushTask", ImmutableMap.of("schedule", "0 */20 * ? * * *")))).build();
+    updateTableConfig(tableConfig);
+    Thread.sleep(2000);
+    jobGroupNames = scheduler.getJobGroupNames();
+    Assert.assertEquals(jobGroupNames.size(), 1);
+    for (String group : jobGroupNames) {
+      Set<JobKey> jobKeys = scheduler.getJobKeys(GroupMatcher.groupEquals(group));
+      for (JobKey jobKey : jobKeys) {
+        JobDetail jobDetail = scheduler.getJobDetail(jobKey);
+        Assert.assertEquals(jobDetail.getJobClass(), CronJobScheduleJob.class);
+        Assert.assertEquals(jobDetail.getKey().getName(), RAW_TABLE_NAME + "_OFFLINE");
+        Assert.assertEquals(jobDetail.getKey().getGroup(), MinionConstants.SegmentGenerationAndPushTask.TASK_TYPE);
+        Assert.assertEquals(jobDetail.getJobDataMap().get("PinotTaskManager"), taskManager);
+        Assert.assertEquals(jobDetail.getJobDataMap().get("LeadControllerManager"),
+            _controllerStarter.getLeadControllerManager());
+        List<? extends Trigger> triggersOfJob = scheduler.getTriggersOfJob(jobKey);
+        Assert.assertEquals(triggersOfJob.size(), 1);
+        for (Trigger trigger : triggersOfJob) {
+          Assert.assertTrue(trigger instanceof CronTrigger);
+          Assert.assertEquals(((CronTrigger) trigger).getCronExpression(), "0 */20 * ? * * *");
+        }
+      }
+    }
+    // 3. Drop table
+    dropOfflineTable(RAW_TABLE_NAME);
+    jobGroupNames = scheduler.getJobGroupNames();
+    Assert.assertTrue(jobGroupNames.isEmpty());
+  }
+
+  @AfterClass
+  public void teardown() {
+    stopController();
+    stopZk();
+  }
+}

Review comment:
       (nit) new line




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

codecov-io edited a comment on pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#issuecomment-762195257


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=h1) Report
   > Merging [#6451](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=desc) (7b0bed2) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **increase** coverage by `6.90%`.
   > The diff coverage is `72.97%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6451/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6451      +/-   ##
   ==========================================
   + Coverage   66.44%   73.35%   +6.90%     
   ==========================================
     Files        1075     1332     +257     
     Lines       54773    65226   +10453     
     Branches     8168     9505    +1337     
   ==========================================
   + Hits        36396    47845   +11449     
   + Misses      15700    14247    -1453     
   - Partials     2677     3134     +457     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `43.78% <38.51%> (?)` | |
   | unittests | `65.02% <56.80%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `44.44% <0.00%> (-4.40%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `78.57% <ø> (+5.40%)` | :arrow_up: |
   | [...common/config/tuner/NoOpTableTableConfigTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL05vT3BUYWJsZVRhYmxlQ29uZmlnVHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ot/common/config/tuner/RealTimeAutoIndexTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL1JlYWxUaW1lQXV0b0luZGV4VHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../common/config/tuner/TableConfigTunerRegistry.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL1RhYmxlQ29uZmlnVHVuZXJSZWdpc3RyeS5qYXZh) | `72.00% <ø> (ø)` | |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `100.00% <ø> (ø)` | |
   | ... and [1128 more](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=footer). Last update [28882ba...7b0bed2](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io removed a comment on pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

codecov-io removed a comment on pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#issuecomment-762195257


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=h1) Report
   > Merging [#6451](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=desc) (6a0d78b) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `22.74%`.
   > The diff coverage is `38.51%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6451/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #6451       +/-   ##
   ===========================================
   - Coverage   66.44%   43.70%   -22.75%     
   ===========================================
     Files        1075     1332      +257     
     Lines       54773    65236    +10463     
     Branches     8168     9506     +1338     
   ===========================================
   - Hits        36396    28510     -7886     
   - Misses      15700    34341    +18641     
   + Partials     2677     2385      -292     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `43.70% <38.51%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...org/apache/pinot/broker/queryquota/HitCounter.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9IaXRDb3VudGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...che/pinot/broker/queryquota/MaxHitRateTracker.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9NYXhIaXRSYXRlVHJhY2tlci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ache/pinot/broker/queryquota/QueryQuotaEntity.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9RdWVyeVF1b3RhRW50aXR5LmphdmE=) | `0.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ceselector/StrictReplicaGroupInstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL1N0cmljdFJlcGxpY2FHcm91cEluc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...roker/routing/segmentpruner/TimeSegmentPruner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL1RpbWVTZWdtZW50UHJ1bmVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...roker/routing/segmentpruner/interval/Interval.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL2ludGVydmFsL0ludGVydmFsLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...r/routing/segmentpruner/interval/IntervalTree.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL2ludGVydmFsL0ludGVydmFsVHJlZS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | ... and [1321 more](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=footer). Last update [28882ba...6a0d78b](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io removed a comment on pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

codecov-io removed a comment on pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#issuecomment-761898819


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=h1) Report
   > Merging [#6451](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=desc) (e2d607e) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.40%`.
   > The diff coverage is `56.80%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6451/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6451      +/-   ##
   ==========================================
   - Coverage   66.44%   65.04%   -1.41%     
   ==========================================
     Files        1075     1332     +257     
     Lines       54773    65187   +10414     
     Branches     8168     9498    +1330     
   ==========================================
   + Hits        36396    42398    +6002     
   - Misses      15700    19752    +4052     
   - Partials     2677     3037     +360     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.04% <56.80%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [...common/config/tuner/NoOpTableTableConfigTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL05vT3BUYWJsZVRhYmxlQ29uZmlnVHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ot/common/config/tuner/RealTimeAutoIndexTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL1JlYWxUaW1lQXV0b0luZGV4VHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | ... and [1173 more](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=footer). Last update [28882ba...e2d607e](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

codecov-io edited a comment on pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#issuecomment-761898819


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=h1) Report
   > Merging [#6451](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=desc) (fa192a8) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **increase** coverage by `6.94%`.
   > The diff coverage is `72.97%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6451/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6451      +/-   ##
   ==========================================
   + Coverage   66.44%   73.39%   +6.94%     
   ==========================================
     Files        1075     1332     +257     
     Lines       54773    65162   +10389     
     Branches     8168     9502    +1334     
   ==========================================
   + Hits        36396    47825   +11429     
   + Misses      15700    14212    -1488     
   - Partials     2677     3125     +448     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `43.83% <38.60%> (?)` | |
   | unittests | `65.07% <56.80%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `44.44% <0.00%> (-4.40%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `78.57% <ø> (+5.40%)` | :arrow_up: |
   | [...common/config/tuner/NoOpTableTableConfigTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL05vT3BUYWJsZVRhYmxlQ29uZmlnVHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ot/common/config/tuner/RealTimeAutoIndexTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL1JlYWxUaW1lQXV0b0luZGV4VHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../common/config/tuner/TableConfigTunerRegistry.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL1RhYmxlQ29uZmlnVHVuZXJSZWdpc3RyeS5qYXZh) | `72.00% <ø> (ø)` | |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `100.00% <ø> (ø)` | |
   | ... and [1120 more](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=footer). Last update [832ece7...fa192a8](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io commented on pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

codecov-io commented on pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#issuecomment-762195257


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=h1) Report
   > Merging [#6451](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=desc) (946f580) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `22.66%`.
   > The diff coverage is `38.51%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6451/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #6451       +/-   ##
   ===========================================
   - Coverage   66.44%   43.78%   -22.67%     
   ===========================================
     Files        1075     1332      +257     
     Lines       54773    65215    +10442     
     Branches     8168     9505     +1337     
   ===========================================
   - Hits        36396    28552     -7844     
   - Misses      15700    34265    +18565     
   + Partials     2677     2398      -279     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `43.78% <38.51%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...org/apache/pinot/broker/queryquota/HitCounter.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9IaXRDb3VudGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...che/pinot/broker/queryquota/MaxHitRateTracker.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9NYXhIaXRSYXRlVHJhY2tlci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ache/pinot/broker/queryquota/QueryQuotaEntity.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9RdWVyeVF1b3RhRW50aXR5LmphdmE=) | `0.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ceselector/StrictReplicaGroupInstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL1N0cmljdFJlcGxpY2FHcm91cEluc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...roker/routing/segmentpruner/TimeSegmentPruner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL1RpbWVTZWdtZW50UHJ1bmVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...roker/routing/segmentpruner/interval/Interval.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL2ludGVydmFsL0ludGVydmFsLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...r/routing/segmentpruner/interval/IntervalTree.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL2ludGVydmFsL0ludGVydmFsVHJlZS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | ... and [1323 more](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=footer). Last update [28882ba...946f580](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6451: Adding CRON scheduler for Pinot tasks

Posted by GitBox <gi...@apache.org>.

codecov-io edited a comment on pull request #6451:
URL: https://github.com/apache/incubator-pinot/pull/6451#issuecomment-762195257


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=h1) Report
   > Merging [#6451](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=desc) (7b0bed2) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.42%`.
   > The diff coverage is `56.80%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6451/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6451      +/-   ##
   ==========================================
   - Coverage   66.44%   65.02%   -1.43%     
   ==========================================
     Files        1075     1332     +257     
     Lines       54773    65226   +10453     
     Branches     8168     9505    +1337     
   ==========================================
   + Hits        36396    42411    +6015     
   - Misses      15700    19777    +4077     
   - Partials     2677     3038     +361     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.02% <56.80%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [...common/config/tuner/NoOpTableTableConfigTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL05vT3BUYWJsZVRhYmxlQ29uZmlnVHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ot/common/config/tuner/RealTimeAutoIndexTuner.java](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3R1bmVyL1JlYWxUaW1lQXV0b0luZGV4VHVuZXIuamF2YQ==) | `100.00% <ø> (ø)` | |
   | ... and [1172 more](https://codecov.io/gh/apache/incubator-pinot/pull/6451/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=footer). Last update [28882ba...7b0bed2](https://codecov.io/gh/apache/incubator-pinot/pull/6451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org