You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Copilot (via GitHub)" <gi...@apache.org> on 2026/02/06 16:55:38 UTC
Re: [PR] [test] Fix flaky table cleanup and truncate signed-zero tests [pinot]
Copilot commented on code in PR #17652:
URL: https://github.com/apache/pinot/pull/17652#discussion_r2775102945
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TruncateDecimalTransformFunction.java:
##########
@@ -98,7 +98,8 @@ public double[] transformToDoubleValuesSV(ValueBlock valueBlock) {
}
} else {
for (int i = 0; i < length; i++) {
- _doubleValuesSV[i] = Math.signum(leftValues[i]) * Math.floor(Math.abs(leftValues[i]));
+ // Keep behavior consistent with truncate(value, 0), including canonical +0.0 for values truncated to zero.
+ _doubleValuesSV[i] = BigDecimal.valueOf(leftValues[i]).setScale(0, RoundingMode.DOWN).doubleValue();
Review Comment:
This changes behavior for non-finite doubles: `BigDecimal.valueOf(Double.NaN/Infinity)` throws `NumberFormatException`, whereas the previous `Math.signum/floor/abs` logic returned `NaN`/`Infinity`. If `truncate()` can be applied to columns containing non-finite values, this is a runtime regression. Consider preserving prior semantics by explicitly handling non-finite inputs (e.g., if not finite, return the original value) before using `BigDecimal`.
##########
pinot-controller/src/test/java/org/apache/pinot/controller/api/PinotTableRestletResourceTest.java:
##########
@@ -1197,18 +1197,21 @@ public void testTableTasksCleanupWithNonActiveTasks()
String taskName = taskInfo.values().iterator().next().getScheduledTaskNames().get(0);
waitForTaskState(taskName, TaskState.IN_PROGRESS);
- // stop the task queue to abort the task
+ // Stop the task queue to abort the task. Keep it stopped until table deletion is complete to avoid
+ // task transition races while cleanup deletes Helix job metadata.
sendPutRequest(DEFAULT_INSTANCE.getControllerRequestURLBuilder()
.forStopMinionTaskQueue(MinionConstants.SegmentGenerationAndPushTask.TASK_TYPE));
waitForTaskState(taskName, TaskState.STOPPED);
- // resume the task queue again to avoid affecting other tests
- sendPutRequest(DEFAULT_INSTANCE.getControllerRequestURLBuilder()
- .forResumeMinionTaskQueue(MinionConstants.SegmentGenerationAndPushTask.TASK_TYPE));
-
- // Delete table - should succeed and clean up tasks
- String deleteResponse = sendDeleteRequest(
- DEFAULT_INSTANCE.getControllerRequestURLBuilder().forTableDelete(tableName));
- assertEquals(deleteResponse, "{\"status\":\"Tables: [" + tableName + "_OFFLINE] deleted\"}");
+ try {
+ // Delete table - should succeed and clean up tasks
+ String deleteResponse = sendDeleteRequest(
+ DEFAULT_INSTANCE.getControllerRequestURLBuilder().forTableDelete(tableName));
+ assertEquals(deleteResponse, "{\"status\":\"Tables: [" + tableName + "_OFFLINE] deleted\"}");
+ } finally {
+ // Resume queue to avoid affecting other tests.
+ sendPutRequest(DEFAULT_INSTANCE.getControllerRequestURLBuilder()
+ .forResumeMinionTaskQueue(MinionConstants.SegmentGenerationAndPushTask.TASK_TYPE));
+ }
Review Comment:
The `finally` only protects the table deletion section. If `waitForTaskState(taskName, TaskState.STOPPED)` fails/throws (e.g., timeout), the queue may remain stopped and leak state to other tests—the exact scenario the `finally` is meant to prevent. Consider widening the `try/finally` to cover everything after the stop request (including the STOPPED wait), so the resume always runs once a stop has been issued.
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TruncateDecimalTransformFunction.java:
##########
@@ -98,7 +98,8 @@ public double[] transformToDoubleValuesSV(ValueBlock valueBlock) {
}
} else {
for (int i = 0; i < length; i++) {
- _doubleValuesSV[i] = Math.signum(leftValues[i]) * Math.floor(Math.abs(leftValues[i]));
+ // Keep behavior consistent with truncate(value, 0), including canonical +0.0 for values truncated to zero.
+ _doubleValuesSV[i] = BigDecimal.valueOf(leftValues[i]).setScale(0, RoundingMode.DOWN).doubleValue();
Review Comment:
Creating a `BigDecimal` per row in `transformToDoubleValuesSV()` can be expensive in hot query paths (allocations + parsing). Since the main goal is canonicalizing signed zero, consider a math-based truncation (toward zero) and then normalize zero explicitly (e.g., if truncated == 0.0d then return 0.0d) to avoid per-row `BigDecimal` overhead.
```suggestion
double value = leftValues[i];
double truncated;
if (value > 0.0d) {
truncated = Math.floor(value);
} else if (value < 0.0d) {
truncated = Math.ceil(value);
} else {
truncated = 0.0d;
}
// Normalize -0.0 to +0.0 to match BigDecimal behavior.
if (truncated == 0.0d) {
truncated = 0.0d;
}
_doubleValuesSV[i] = truncated;
```
##########
pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/TruncateDecimalTransformFunctionTest.java:
##########
@@ -62,6 +62,15 @@ public void testTruncateDecimalTransformFunction() {
expectedValues[i] = truncate(_doubleSVValues[i], 0);
}
testTransformFunction(transformFunction, expectedValues);
+
+ // Regression for signed-zero handling: truncate(value) should match truncate(value, 0).
+ expression = RequestContextUtils.getExpression("truncate(-0.4)");
+ transformFunction = TransformFunctionFactory.get(expression, _dataSourceMap);
+ Assert.assertTrue(transformFunction instanceof TruncateDecimalTransformFunction);
+ for (int i = 0; i < NUM_ROWS; i++) {
+ expectedValues[i] = 0.0d;
+ }
Review Comment:
This loop is filling the entire array with a constant value; using `Arrays.fill(expectedValues, 0.0d)` would be simpler and clearer, and avoids repeating the same fill-loop pattern in tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org