You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "ayesheepatra07 (via GitHub)" <gi...@apache.org> on 2025/04/04 19:07:47 UTC
[PR] Fallback for bad values in periodic controller task configurations [pinot]
ayesheepatra07 opened a new pull request, #15466:
URL: https://github.com/apache/pinot/pull/15466
This change improves how the controller conf applies the values for periodic task configurations. Previously, invalid new configuration values were throwing exceptions and not letting the service startup. With this update, we now: Validate that the configured period strings (like 7200s, 2h) pass the validation check in TimeUtils.isPeriodValid(). Log an error message when an invalid period string is configured, and the fallback to the default deprecated old configuration. Continue to fallback to default deprecated old configuration values when new configuration is not provided. These are configurations that are updated in this change: getRetentionControllerFrequencyInSeconds()
getOfflineSegmentIntervalCheckerFrequencyInSeconds() getRealtimeSegmentValidationFrequencyInSeconds()
getBrokerResourceValidationFrequencyInSeconds()
getStatusCheckerFrequencyInSeconds()
getStatusCheckerWaitForPushTimeInSeconds()
getTaskMetricsEmitterFrequencyInSeconds()
getSegmentLevelValidationIntervalInSeconds()
getMinionInstancesCleanupTaskFrequencyInSeconds()
getMinionInstancesCleanupTaskMinOfflineTimeBeforeDeletionInSeconds() getStaleInstancesCleanupTaskFrequencyInSeconds()
getStaleInstancesCleanupTaskInstancesRetentionInSeconds()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fallback for bad values in periodic controller task configurations [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang merged PR #15466:
URL: https://github.com/apache/pinot/pull/15466
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fallback for bad values in periodic controller task configurations [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #15466:
URL: https://github.com/apache/pinot/pull/15466#discussion_r2072235191
##########
pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java:
##########
@@ -367,6 +367,8 @@ private static long getRandomInitialDelayInSeconds() {
public static final String EXIT_ON_SCHEMA_CHECK_FAILURE = "controller.startup.exitOnSchemaCheckFailure";
public static final boolean DEFAULT_EXIT_ON_SCHEMA_CHECK_FAILURE = true;
+ private final Map<String, String> _invalidConfigs = new HashMap<>();
Review Comment:
We need a concurrent map here for thread safety
##########
pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java:
##########
@@ -1049,12 +1078,15 @@ public boolean isHybridTableRetentionStrategyEnabled() {
return getProperty(ENABLE_HYBRID_TABLE_RETENTION_STRATEGY, DEFAULT_ENABLE_HYBRID_TABLE_RETENTION_STRATEGY);
}
- public int getSegmentLevelValidationIntervalInSeconds() {
- return Optional.ofNullable(getProperty(ControllerPeriodicTasksConf.SEGMENT_LEVEL_VALIDATION_INTERVAL_PERIOD))
- .map(period -> (int) convertPeriodToSeconds(period)).orElseGet(
- () -> getProperty(ControllerPeriodicTasksConf.DEPRECATED_SEGMENT_LEVEL_VALIDATION_INTERVAL_IN_SECONDS,
- ControllerPeriodicTasksConf.DEFAULT_SEGMENT_LEVEL_VALIDATION_INTERVAL_IN_SECONDS));
- }
+ public int getSegmentLevelValidationIntervalInSeconds() {
+ return Optional.ofNullable(getProperty(ControllerPeriodicTasksConf.SEGMENT_LEVEL_VALIDATION_INTERVAL_PERIOD))
+ .filter(period -> isValidPeriodWithLogging(
+ ControllerPeriodicTasksConf.SEGMENT_LEVEL_VALIDATION_INTERVAL_PERIOD, period))
+ .map(period -> (int) convertPeriodToSeconds(period)).orElseGet(
+ () -> getProperty(ControllerPeriodicTasksConf.DEPRECATED_SEGMENT_LEVEL_VALIDATION_INTERVAL_IN_SECONDS,
+ ControllerPeriodicTasksConf.DEFAULT_SEGMENT_LEVEL_VALIDATION_INTERVAL_IN_SECONDS));
+ }
Review Comment:
(minor) Indentation seems wrong
##########
pinot-controller/src/test/java/org/apache/pinot/controller/ControllerConfTest.java:
##########
@@ -88,9 +88,10 @@ public void validNewConfigOverridesCorrespondingValidOrInvalidOldConfigOnRead()
* are thrown when invalid new configurations are read (there is no fall-back to the corresponding
* valid deprecated configuration). For all valid new configurations, they override the
* corresponding deprecated configuration.
+ * Added fallback logic to use valid deprecated config when new config is invalid.
*/
@Test
- public void invalidNewConfigShouldThrowExceptionOnReadWithoutFallbackToCorrespondingValidDeprecatedConfig() {
+ public void invalidNewConfigShouldNotThrowExceptionOnReadWithFallbackToCorrespondingValidDeprecatedConfig() {
Review Comment:
Can we also add a test for the invalid configs map value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fallback for bad values in periodic controller task configurations [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #15466:
URL: https://github.com/apache/pinot/pull/15466#issuecomment-2779615101
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15466?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 62.98%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`39c5682`)](https://app.codecov.io/gh/apache/pinot/commit/39c56828066ea57457bcf9e8bafcf4a4c2026896?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 1992 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #15466 +/- ##
============================================
+ Coverage 61.75% 62.98% +1.22%
- Complexity 207 1365 +1158
============================================
Files 2436 2835 +399
Lines 133233 160418 +27185
Branches 20636 24580 +3944
============================================
+ Hits 82274 101033 +18759
- Misses 44911 51756 +6845
- Partials 6048 7629 +1581
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.98% <100.00%> (+1.35%)` | :arrow_up: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.96% <100.00%> (+1.22%)` | :arrow_up: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.94% <100.00%> (+35.21%)` | :arrow_up: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.98% <100.00%> (+1.22%)` | :arrow_up: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.98% <100.00%> (+1.23%)` | :arrow_up: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.02% <ø> (+9.13%)` | :arrow_up: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/15466/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.60% <100.00%> (+5.87%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/15466?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fallback for bad values in periodic controller task configurations [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #15466:
URL: https://github.com/apache/pinot/pull/15466#discussion_r2032007475
##########
pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java:
##########
@@ -1200,6 +1235,16 @@ private long convertPeriodToSeconds(String period) {
return convertPeriodToUnit(period, TimeUnit.SECONDS);
}
+ private boolean isValidPeriodWithLogging(String propertyKey, String periodStr) {
+ if (TimeUtils.isPeriodValid(periodStr)) {
+ return true;
+ } else {
+ LOGGER.error("Invalid time spec '{}' for config '{}'. Falling back to default config.", periodStr,
Review Comment:
Logging an error is not very user friendly because a lot of people don't really check logs. We should store these errors in the service starter, and hook them up with the UI so that user knows these configs are not picked up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fallback for bad values in periodic controller task configurations [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #15466:
URL: https://github.com/apache/pinot/pull/15466#discussion_r2048000600
##########
pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java:
##########
@@ -367,6 +372,8 @@ private static long getRandomInitialDelayInSeconds() {
public static final String EXIT_ON_SCHEMA_CHECK_FAILURE = "controller.startup.exitOnSchemaCheckFailure";
public static final boolean DEFAULT_EXIT_ON_SCHEMA_CHECK_FAILURE = true;
+ private final Map<String, String> _warnings = new HashMap<>();
Review Comment:
Suggest making it more specific, such as `invalidConfigs`
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotControllerHealthCheck.java:
##########
@@ -119,6 +121,20 @@ public long getUptime() {
return uptime.getSeconds();
}
+ @GET
+ @Path("/pinot-controller/invalid/configs")
+ @Authorize(targetType = TargetType.CLUSTER, action = Actions.Cluster.GET_HEALTH)
+ @Produces(MediaType.APPLICATION_JSON)
+ @ApiOperation(value = "List cluster config warnings")
Review Comment:
It only lists the invalid controller configs
##########
pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java:
##########
@@ -87,6 +89,9 @@ public enum ControllerMode {
DUAL, PINOT_ONLY, HELIX_ONLY
}
+ private static final Logger LOGGER = LoggerFactory.getLogger(ControllerConf.class);
+
+
Review Comment:
Seems not used
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -263,7 +263,6 @@ public void init(PinotConfiguration pinotConfiguration)
"tenant-rebalance-thread-%d");
_tenantRebalancer = new DefaultTenantRebalancer(_helixResourceManager, _tenantRebalanceExecutorService);
}
-
Review Comment:
(nit) Revert
##########
pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java:
##########
@@ -1200,6 +1237,26 @@ private long convertPeriodToSeconds(String period) {
return convertPeriodToUnit(period, TimeUnit.SECONDS);
}
+ private boolean isValidPeriodWithLogging(String propertyKey, String periodStr) {
+ if (TimeUtils.isPeriodValid(periodStr)) {
+ return true;
+ } else {
+ String message = String.format(
+ "Invalid time spec '%s' for config '%s'. Falling back to default config.", periodStr,
+ propertyKey);
+ addControllerInvalidConfigs(propertyKey, message);
+ return false;
+ }
+ }
+
+ private void addControllerInvalidConfigs(String propertyKey, String errorMessage) {
+ _warnings.put(propertyKey, errorMessage);
+ }
+
+ public Map<String, String> getControllerInvalidConfigs() {
Review Comment:
(minor)
```suggestion
public Map<String, String> getInvalidConfigs() {
```
##########
pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java:
##########
@@ -1200,6 +1237,26 @@ private long convertPeriodToSeconds(String period) {
return convertPeriodToUnit(period, TimeUnit.SECONDS);
}
+ private boolean isValidPeriodWithLogging(String propertyKey, String periodStr) {
+ if (TimeUtils.isPeriodValid(periodStr)) {
+ return true;
+ } else {
+ String message = String.format(
+ "Invalid time spec '%s' for config '%s'. Falling back to default config.", periodStr,
+ propertyKey);
+ addControllerInvalidConfigs(propertyKey, message);
Review Comment:
(minor) This can be inlined
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotControllerHealthCheck.java:
##########
@@ -119,6 +121,20 @@ public long getUptime() {
return uptime.getSeconds();
}
+ @GET
+ @Path("/pinot-controller/invalid/configs")
Review Comment:
This is an API on controller, so we can simplify it to
```suggestion
@Path("invalidconfigs")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org