You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/22 23:08:33 UTC

[GitHub] [spark] FridaPolished opened a new pull request #32304: [SPARK-35197][Web UI]

FridaPolished opened a new pull request #32304:
URL: https://github.com/apache/spark/pull/32304

**What changes were proposed in this pull request?**
The original Spark UI on SHS does not have a specific way to explore accumulators information, this data is scattered across stages which makes it difficult to access it.
The accumulators information is located on the Stage Page, but this column is hard to read since it groups all the accumulators names and values for a specific task. This is one of the reasons why understanding the changes of counters from a job can become a difficult task.
The proposed change creates an explore page for Accumulators on the SparkUI. The information will be located under a new tab that has an overview page with links to check for more details about the accumulators information by a particular name or stage.
[Community discussion on accumulators page](http://apache-spark-developers-list.1001551.n3.nabble.com/accumulators-td8814.html)

**Why are the changes needed?**
A page that allows users to explore accumulators information is a helpful feature for users because currently the information is scattered across stages and the display on the Stages Page is difficult to read. This page offers easy access to accumulators information in detail which translates into a better user experience and an increased velocity to solve problems and track accumulator changes.

**Does this PR introduce user-facing change?**
Yes. A new tab will be added to the Spark UI and three new pages will dependent on it.

**How was this patch tested?**
This patch was tested manually by using real spark event logs from a Spark History Server on a production cluster. I also used unit testing to check for contents inside the new data structure used for the Accumulators Overview page.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] github-actions[bot] closed pull request #32304: [SPARK-35197][Web UI] Accumulators Explore Page on Spark UI on History Server

Posted by GitBox <gi...@apache.org>.

github-actions[bot] closed pull request #32304:
URL: https://github.com/apache/spark/pull/32304


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #32304: [SPARK-35197][Web UI]

Posted by GitBox <gi...@apache.org>.

HyukjinKwon commented on pull request #32304:
URL: https://github.com/apache/spark/pull/32304#issuecomment-825527108


   @FridaPolished, can you edit the PR title to be complete to say what the PR proposes?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] github-actions[bot] commented on pull request #32304: [SPARK-35197][Web UI] Accumulators Explore Page on Spark UI on History Server

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on pull request #32304:
URL: https://github.com/apache/spark/pull/32304#issuecomment-890613509


   We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
   If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #32304: [SPARK-35197][Web UI]

Posted by GitBox <gi...@apache.org>.

AmplabJenkins commented on pull request #32304:
URL: https://github.com/apache/spark/pull/32304#issuecomment-825245346


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org