Re: [I] Memory account not adding up in SortExec [datafusion]

["yjshen (via GitHub)" <gi...@apache.org>]

yjshen commented on issue #10073:
URL: https://github.com/apache/datafusion/issues/10073#issuecomment-2308963109

   > Through examining the current implementation of multi-column sort's spill-to-disk strategies, I find we are asking for more memory during spill, which I think is worth discussing:
   During the spill, Rows are created for comparison efficiency for each in-memory RecordBatch. Considering why we spill in the first place, does this Rows optimization increase the possibility of execution failure due to memory shortage?
   
   I find this also related. https://github.com/apache/datafusion/discussions/9528#discussion-6349150


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org