[I] [VL] Support GPU shuffle reader [incubator-gluten]

["jinchengchenghh (via GitHub)" <gi...@apache.org>]

jinchengchenghh opened a new issue, #10933:
URL: https://github.com/apache/incubator-gluten/issues/10933

   ### Description
   
   1. Move the lock from WholeStageResultIterator constructor to shuffle reader, the threads can prepare the first batch in advance, now it is a small batch, for GPU, it will produce a big batch with bytes 1GB
   2. Implement the decompression, convert  buffer to cudf table, resize batch in GPU
   3. Then the threads can prepare read buffer from file.
   
   ### Gluten version
   
   None


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Support GPU shuffle reader [incubator-gluten]

["jinchengchenghh (via GitHub)" <gi...@apache.org>]

jinchengchenghh commented on issue #10933:
URL: https://github.com/apache/incubator-gluten/issues/10933#issuecomment-3490755810

   In CPU Gluten, we uses VeloxResizesBatchesExec to concat the batch after shuffle reader, this can help concat from different stream readers VS concat in ShuffleReader.
   But in GPU gluten, if we split to cudf::table and concatenateTables, most of the concatenateTables operation is CPU work, because we add the GPU lock to restrict GPU threads before generating cudf::table, so we should do the concat in CPU side.
   
   The lock is used for limited GPU memory. So most time the number of GPU threads should be less than CPU threads.
   
   For example, the EC2 instance g4dn.2xlarge, CPU memory is 30G while GPU memory is 15G.
   
   Future work:
   - Operator VeloxBuffersBatchExec, save all the buffers to std::vector, wrap it to new class BufferColumnarBatch. For example, the flat Vector contains {BufferPtr nulls, BufferPtr values}, not it is {std::vector<BufferPtr> nulls, std::vector<BufferPtr> values}
   - The first thread uses GPU decompression while the other threads still uses CPU decompression, because the other threads cannot execute in GPU immediately, they need to wait for GPU execution.
   - The other threads fetch the BufferColumnarBatch and concat to big batch continuously util there is not enough memory, save the Batches to std::vector<ColumnarBatch> and wait the downstream operator to fetch.
   - After decompression, merge the buffer especially for the validity buffer which requires ByteForBits, different copy, and also different for the offset buffer, needs to recompute the offset.
   - Lock and construct cudf::table
   
   Question:
   
   Is the first thread GPU decompression make senses? Looks like complex and may not benefit because the concat buffer is in single GPU thread in these cases.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org