Re: [I] Transformations using columns which themselves are a result of transformation [pinot]

["xiangfu0 (via GitHub)" <gi...@apache.org>]

xiangfu0 commented on issue #5351:
URL: https://github.com/apache/pinot/issues/5351#issuecomment-4552760723

   Confirmed this is already supported on current `master`.
   
   The ingestion `ExpressionTransformer` topologically sorts transform evaluators before applying them, so a transform output can be used by another transform in the same config: https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ExpressionTransformer.java#L95-L157
   
   There are focused tests covering this behavior:
   - `testTransformFunctionSortOrder` chains `d = plus(x, 10)`, `b = plus(d, 10)`, `a = plus(b, 10)`, `c = plus(a, d)`: https://github.com/apache/pinot/blob/master/pinot-segment-local/src/test/java/org/apache/pinot/segment/local/recordtransformer/ExpressionTransformerTest.java#L358-L385
   - `testNonCyclicTransformFunctionSortOrder` verifies dependency ordering: https://github.com/apache/pinot/blob/master/pinot-segment-local/src/test/java/org/apache/pinot/segment/local/recordtransformer/ExpressionTransformerTest.java#L408-L434
   
   The original support was merged in #6495 (`Support chained transform functions`) on 2021-01-27. I also verified current `master` (`baccdcc489017df4449f5e594d3fab564b504ab2`) with:
   
   ```bash
   ./mvnw -pl pinot-segment-local -Dtest=ExpressionTransformerTest#testTransformFunctionSortOrder+testNonCyclicTransformFunctionSortOrder test
   ```
   
   Result: `BUILD SUCCESS`, 2 tests run, 0 failures/errors.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org