Re: [PR] Fix bytes pk for dim tables [pinot]

["Jackie-Jiang (via GitHub)" <gi...@apache.org>]

Jackie-Jiang commented on code in PR #19501:
URL: https://github.com/apache/pinot/pull/19501#discussion_r4186585296


##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/offline/DimensionTableDataManager.java:
##########
@@ -317,7 +313,7 @@ private DimensionTable createMemOptimisedDimensionTable() {
               return null;
             }
 
-            Object[] primaryKey = recordReader.getRecordValues(i, pkIndexes);
+            Object[] primaryKey = recordReader.getPrimaryKeys(i);

Review Comment:
   [P1] Normalize BYTES results for memory-optimized lookup joins
   
   With `disablePreload=true`, BYTES probes now match, but returning the dimension PK through an MSE lookup join still fails. `MemoryOptimizedDimensionTable.getValues()` reads it through `PinotSegmentRecordReader.getValue()`, which returns `byte[]`. `LookupJoinOperator` copies that value into its result unchanged, while `DataBlockBuilder.buildFromRows()` casts BYTES values to `ByteArray`, causing `ClassCastException` during serialization. This leaves the BYTES fix incomplete for this mode. Please normalize results at the MSE boundary and extend the new join regression to both preload settings (including a LEFT JOIN that projects the dimension PK); the new fixture currently hardcodes `disablePreload=false`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org