You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Bjorn Jonsson (JIRA)" <ji...@apache.org> on 2019/05/30 00:41:00 UTC

[jira] [Created] (SPARK-27874) ShuffleBlockFetchIterator can log information about per-block sizes

Bjorn Jonsson created SPARK-27874:
-------------------------------------

             Summary: ShuffleBlockFetchIterator can log information about per-block sizes
                 Key: SPARK-27874
                 URL: https://issues.apache.org/jira/browse/SPARK-27874
             Project: Spark
          Issue Type: Improvement
          Components: Shuffle, Spark Core
    Affects Versions: 2.4.2
            Reporter: Bjorn Jonsson


It can be beneficial for monitoring and debugging to know the size of the largest shuffle block (or some distribution of block sizes). The ShuffleBlockFetcherIterator currently logs the number of local & remote blocks and their total size (see SPARK-26697), but does not log any information about per-block sizes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org