You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/23 16:13:16 UTC

[GitHub] [arrow-datafusion] andygrove commented on pull request #398: Add metrics to RepartitionExec

andygrove commented on pull request #398:
URL: https://github.com/apache/arrow-datafusion/pull/398#issuecomment-846587539


   > Looks good. I think the time calculation for round robin repartition is missing.
   
   The new metrics don't include the time for sending the resulting batches to the channels, so the only thing to measure for round-robin would be the time to execute `let output_partition = counter % num_output_partitions` so I figured that was not worth measuring.
   
   I am now wondering if we should also measure time to send the results to the channel because if this is high it could indicate that upstream operators are not fetching data as fast as they could be. I will take a look at that next.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org