[jira] [Created] (SPARK-27975) ConsoleSink should display alias and options for streaming progress

["Jacek Laskowski (JIRA)" <ji...@apache.org>]

Jacek Laskowski created SPARK-27975:
---------------------------------------

             Summary: ConsoleSink should display alias and options for streaming progress
                 Key: SPARK-27975
                 URL: https://issues.apache.org/jira/browse/SPARK-27975
             Project: Spark
          Issue Type: Improvement
          Components: Structured Streaming
    Affects Versions: 2.4.3
            Reporter: Jacek Laskowski


{{console}} sink shows itself in progress with this internal instance representation as follows:

{code:json}
  "sink" : {
    "description" : "org.apache.spark.sql.execution.streaming.ConsoleSinkProvider@12fa674a"
  }
{code}

That is not very user-friendly and would be much better for debugging if it included the alias and options as {{socket}} does:

{code}
  "sources" : [ {
    "description" : "TextSocketV2[host: localhost, port: 8888]",
    ...
  } ],
{code}

The entire sample progress looks as follows:

{code}
19/06/07 11:47:18 INFO MicroBatchExecution: Streaming query made progress: {
  "id" : "26bedc9f-076f-4b15-8e17-f09609aaecac",
  "runId" : "8c365e74-7ac9-4fad-bf1b-397eb086661e",
  "name" : "socket-console",
  "timestamp" : "2019-06-07T09:47:18.969Z",
  "batchId" : 2,
  "numInputRows" : 0,
  "inputRowsPerSecond" : 0.0,
  "durationMs" : {
    "getEndOffset" : 0,
    "setOffsetRange" : 0,
    "triggerExecution" : 0
  },
  "stateOperators" : [ ],
  "sources" : [ {
    "description" : "TextSocketV2[host: localhost, port: 8888]",
    "startOffset" : 0,
    "endOffset" : 0,
    "numInputRows" : 0,
    "inputRowsPerSecond" : 0.0
  } ],
  "sink" : {
    "description" : "org.apache.spark.sql.execution.streaming.ConsoleSinkProvider@12fa674a"
  }
}
{code}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org