You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by shahidki31 <gi...@git.apache.org> on 2018/11/29 18:32:52 UTC

[GitHub] spark pull request #23181: Executor summary should update for history events

GitHub user shahidki31 opened a pull request:

    https://github.com/apache/spark/pull/23181

    Executor summary should update for history events

    ## What changes were proposed in this pull request?
    The root cause of the problem is, whenever the taskEnd event comes after stageCompleted event, execSummary is updating only for live UI. we need to update for history UI too.
    
    To see the previous discussion, refer: PR for https://github.com/apache/spark/pull/23038, https://issues.apache.org/jira/browse/SPARK-26100.
    
    ## How was this patch tested?
    
    Added UT. Manually verified


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/shahidki31/spark executorUpdate

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/23181.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #23181
    
----
commit ae71eba254235c6317c91064e9a7b2e55c1c1cce
Author: Shahid <sh...@...>
Date:   2018-11-29T18:09:45Z

    Executor summary should update for history events

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99468/testReport)** for PR 23181 at commit [`1be36f7`](https://github.com/apache/spark/commit/1be36f77f58576db9650a4584b1f882e2f284d0f).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/23181


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Before patch:
    ![screenshot from 2018-11-29 22-13-34](https://user-images.githubusercontent.com/23054875/49246338-a21ead00-f43a-11e8-8214-f1020420be52.png)
    
    After patch:
    ![screenshot from 2018-11-30 00-54-49](https://user-images.githubusercontent.com/23054875/49246353-aa76e800-f43a-11e8-98ef-7faecaa7a50e.png)



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/23181#discussion_r237952479
  
    --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala ---
    @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends SparkFunSuite with BeforeAndAfter {
       }
     
       test("SPARK-25451: total tasks in the executor summary should match total stage tasks") {
    -    val testConf = conf.clone.set(LIVE_ENTITY_UPDATE_PERIOD, Long.MaxValue)
     
    -    val listener = new AppStatusListener(store, testConf, true)
    +    val isLiveSeq = Seq(true, false)
     
    -    val stage = new StageInfo(1, 0, "stage", 4, Nil, Nil, "details")
    -    listener.onJobStart(SparkListenerJobStart(1, time, Seq(stage), null))
    -    listener.onStageSubmitted(SparkListenerStageSubmitted(stage, new Properties()))
    +    isLiveSeq.foreach { live: Boolean =>
    --- End diff --
    
    When doing things like this I prefer to invert the logic.
    
    ```
    Seq(true, false).foreach { live =>
      test(s"blah blah blah (live = $live)") {
    
      }
    }
    ```


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99482/testReport)** for PR 23181 at commit [`1be36f7`](https://github.com/apache/spark/commit/1be36f77f58576db9650a4584b1f882e2f284d0f).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5587/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5537/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    This didn't merge cleanly to 2.4, please open a PR against that branch if you want it there.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99492 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99492/testReport)** for PR 23181 at commit [`1be36f7`](https://github.com/apache/spark/commit/1be36f77f58576db9650a4584b1f882e2f284d0f).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merging to master / 2.4.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/23181#discussion_r237952522
  
    --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala ---
    @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends SparkFunSuite with BeforeAndAfter {
       }
     
       test("SPARK-25451: total tasks in the executor summary should match total stage tasks") {
    -    val testConf = conf.clone.set(LIVE_ENTITY_UPDATE_PERIOD, Long.MaxValue)
     
    -    val listener = new AppStatusListener(store, testConf, true)
    +    val isLiveSeq = Seq(true, false)
     
    -    val stage = new StageInfo(1, 0, "stage", 4, Nil, Nil, "details")
    -    listener.onJobStart(SparkListenerJobStart(1, time, Seq(stage), null))
    -    listener.onStageSubmitted(SparkListenerStageSubmitted(stage, new Properties()))
    +    isLiveSeq.foreach { live: Boolean =>
    +      val testConf = if (live) {
    +        conf.clone.set(LIVE_ENTITY_UPDATE_PERIOD, Long.MaxValue)
    --- End diff --
    
    nit: `clone()`


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99468/testReport)** for PR 23181 at commit [`1be36f7`](https://github.com/apache/spark/commit/1be36f77f58576db9650a4584b1f882e2f284d0f).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99524/testReport)** for PR 23181 at commit [`63ea05a`](https://github.com/apache/spark/commit/63ea05a9e133eb72a46fa181220aa2ef836e8285).
     * This patch **fails PySpark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99492/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5588/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99523/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99523/testReport)** for PR 23181 at commit [`0406b92`](https://github.com/apache/spark/commit/0406b92bcdbacf5984471980c691799a48f66414).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99524/testReport)** for PR 23181 at commit [`63ea05a`](https://github.com/apache/spark/commit/63ea05a9e133eb72a46fa181220aa2ef836e8285).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99492/testReport)** for PR 23181 at commit [`1be36f7`](https://github.com/apache/spark/commit/1be36f77f58576db9650a4584b1f882e2f284d0f).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/23181#discussion_r237959457
  
    --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala ---
    @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends SparkFunSuite with BeforeAndAfter {
       }
     
       test("SPARK-25451: total tasks in the executor summary should match total stage tasks") {
    -    val testConf = conf.clone.set(LIVE_ENTITY_UPDATE_PERIOD, Long.MaxValue)
     
    -    val listener = new AppStatusListener(store, testConf, true)
    +    val isLiveSeq = Seq(true, false)
     
    -    val stage = new StageInfo(1, 0, "stage", 4, Nil, Nil, "details")
    -    listener.onJobStart(SparkListenerJobStart(1, time, Seq(stage), null))
    -    listener.onStageSubmitted(SparkListenerStageSubmitted(stage, new Properties()))
    +    isLiveSeq.foreach { live: Boolean =>
    --- End diff --
    
    Thanks. Updated.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99468/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99482/testReport)** for PR 23181 at commit [`1be36f7`](https://github.com/apache/spark/commit/1be36f77f58576db9650a4584b1f882e2f284d0f).
     * This patch **fails PySpark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99523/testReport)** for PR 23181 at commit [`0406b92`](https://github.com/apache/spark/commit/0406b92bcdbacf5984471980c691799a48f66414).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99466/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Can one of the admins verify this patch?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5548/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99466/testReport)** for PR 23181 at commit [`ae71eba`](https://github.com/apache/spark/commit/ae71eba254235c6317c91064e9a7b2e55c1c1cce).
     * This patch **fails Scala style tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99482/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5558/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Jenkins, retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    **[Test build #99466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99466/testReport)** for PR 23181 at commit [`ae71eba`](https://github.com/apache/spark/commit/ae71eba254235c6317c91064e9a7b2e55c1c1cce).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99524/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Jenkins, retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5536/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Thanks @vanzin. I will open a PR in 2.4 branch


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/23181#discussion_r237959492
  
    --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala ---
    @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends SparkFunSuite with BeforeAndAfter {
       }
     
       test("SPARK-25451: total tasks in the executor summary should match total stage tasks") {
    -    val testConf = conf.clone.set(LIVE_ENTITY_UPDATE_PERIOD, Long.MaxValue)
     
    -    val listener = new AppStatusListener(store, testConf, true)
    +    val isLiveSeq = Seq(true, false)
     
    -    val stage = new StageInfo(1, 0, "stage", 4, Nil, Nil, "details")
    -    listener.onJobStart(SparkListenerJobStart(1, time, Seq(stage), null))
    -    listener.onStageSubmitted(SparkListenerStageSubmitted(stage, new Properties()))
    +    isLiveSeq.foreach { live: Boolean =>
    +      val testConf = if (live) {
    +        conf.clone.set(LIVE_ENTITY_UPDATE_PERIOD, Long.MaxValue)
    --- End diff --
    
    Done


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Jenkins, retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    cc @vanzin Kindly review


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Jenkins, retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/23181
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org