You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "ROHIT KALHANS (Jira)" <ji...@apache.org> on 2019/10/14 08:17:00 UTC

[jira] [Created] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

ROHIT KALHANS created SPARK-29461:
-------------------------------------

             Summary: Spark dataframe writer does not expose metrics for JDBC writer 
                 Key: SPARK-29461
                 URL: https://issues.apache.org/jira/browse/SPARK-29461
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.4.4, 2.4.0
            Reporter: ROHIT KALHANS


Spark does not expose the writer metrics when using the Dataframe JDBC writer. Similar instances of such bugs have been fixed in previous versions. However, it seems the fix was not exhaustive since it does not cover all the writers. 

Similar bugs: 
https://issues.apache.org/jira/browse/SPARK-21882
https://issues.apache.org/jira/browse/SPARK-22605
 
 
Console reporter output 
 app-name.1.executor.bytesWritten
             count = 0 
 
 app-name.1.executor.recordsWritten
             count = 0  
 
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org