You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Ramandeep Singh (JIRA)" <ji...@apache.org> on 2018/11/08 18:40:00 UTC

[jira] [Created] (SPARK-25982) Dataframe write is non blocking in fair scheduling mode

Ramandeep Singh created SPARK-25982:
---------------------------------------

             Summary: Dataframe write is non blocking in fair scheduling mode
                 Key: SPARK-25982
                 URL: https://issues.apache.org/jira/browse/SPARK-25982
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.3.1
            Reporter: Ramandeep Singh


Hi,

I have noticed that expected behavior of dataframe write operation to block is not working in fair scheduling mode.

Ideally when a dataframe write is occurring and a future is blocking on AwaitResult, no other job should be started, but this is not the case. I have noticed that other jobs are started when the partitions are being written.  

 

Regards,

Ramandeep Singh

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org