You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Juliusz Sompolski (JIRA)" <ji...@apache.org> on 2018/02/09 03:48:00 UTC

[jira] [Created] (SPARK-23366) Improve hot reading path in ReadAheadInputStream

Juliusz Sompolski created SPARK-23366:
-----------------------------------------

             Summary: Improve hot reading path in ReadAheadInputStream
                 Key: SPARK-23366
                 URL: https://issues.apache.org/jira/browse/SPARK-23366
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 2.3.0
            Reporter: Juliusz Sompolski


ReadAheadInputStream was introduced in [apache/spark#18317|https://github.com/apache/spark/pull/18317] to optimize reading spill files from disk.
However, investigating flamegraphs of profiles from investigating some regressed workloads after switch to Spark 2.3, it seems that the hot path of reading small amounts of data (like readInt) is inefficient - it involves taking locks, and multiple checks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org