You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2019/10/14 08:40:00 UTC
[jira] [Commented] (SPARK-29461) Spark dataframe writer does not
expose metrics for JDBC writer
[ https://issues.apache.org/jira/browse/SPARK-29461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950808#comment-16950808 ]
Jungtaek Lim commented on SPARK-29461:
--------------------------------------
I'm taking a look at this.
> Spark dataframe writer does not expose metrics for JDBC writer
> ---------------------------------------------------------------
>
> Key: SPARK-29461
> URL: https://issues.apache.org/jira/browse/SPARK-29461
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.4.0, 2.4.4
> Reporter: ROHIT KALHANS
> Priority: Major
>
> Spark does not expose the writer metrics when using the Dataframe JDBC writer. Similar instances of such bugs have been fixed in previous versions. However, it seems the fix was not exhaustive since it does not cover all the writers.
> Similar bugs:
> https://issues.apache.org/jira/browse/SPARK-21882
> https://issues.apache.org/jira/browse/SPARK-22605
>
>
> Console reporter output
> app-name.1.executor.bytesWritten
> count = 0
>
> app-name.1.executor.recordsWritten
> count = 0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org