[GitHub] [spark] xkrogen commented on a change in pull request #30096: [SPARK-33185][YARN] Set up yarn.Client to print direct links to driver stdout/stderr

[GitBox <gi...@apache.org>]

xkrogen commented on a change in pull request #30096:
URL: https://github.com/apache/spark/pull/30096#discussion_r517624169



##########
File path: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
##########
@@ -1172,6 +1189,26 @@ private[spark] class Client(
     }.mkString("")
   }
 
+  /**
+   * Fetch links to the logs of the driver for the given application ID. This requires hitting the
+   * RM REST API. Returns `None` if the links could not be fetched.
+   */
+  private def getDriverLogsLink(appId: ApplicationId): Option[(String, String)] = {
+    val baseRmUrl = WebAppUtils.getRMWebAppURLWithScheme(hadoopConf)
+    val response = ClientBuilder.newClient()
+      .target(baseRmUrl)
+      .path("ws").path("v1").path("cluster").path("apps").path(appId.toString).path("appattempts")
+      .request(MediaType.APPLICATION_JSON)
+      .get()
+    response.getStatusInfo.getFamily match {

Review comment:
       This was a great question. You actually get a valid JSON back with `logsLink` empty, which results in bad behavior. Updating in the new push.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org