You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Takeshi Yamamuro (Jira)" <ji...@apache.org> on 2019/10/25 07:35:00 UTC

[jira] [Resolved] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

     [ https://issues.apache.org/jira/browse/SPARK-29461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Takeshi Yamamuro resolved SPARK-29461.
--------------------------------------
    Fix Version/s: 3.0.0
         Assignee: Jungtaek Lim
       Resolution: Fixed

Resolved by [https://github.com/apache/spark/pull/26109]

> Spark dataframe writer does not expose metrics for JDBC writer 
> ---------------------------------------------------------------
>
>                 Key: SPARK-29461
>                 URL: https://issues.apache.org/jira/browse/SPARK-29461
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: ROHIT KALHANS
>            Assignee: Jungtaek Lim
>            Priority: Minor
>             Fix For: 3.0.0
>
>
> Spark does not expose the writer metrics when using the Dataframe JDBC writer. Similar instances of such bugs have been fixed in previous versions. However, it seems the fix was not exhaustive since it does not cover all the writers. 
> Similar bugs: 
> https://issues.apache.org/jira/browse/SPARK-21882
> https://issues.apache.org/jira/browse/SPARK-22605
>  
>  
> Console reporter output 
>  app-name.1.executor.bytesWritten
>              count = 0 
>  
>  app-name.1.executor.recordsWritten
>              count = 0  
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org