You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by guoxiaolongzte <gi...@git.apache.org> on 2017/10/19 03:23:42 UTC

[GitHub] spark pull request #19532: [CORE]stage api modify the description format, ad...

GitHub user guoxiaolongzte opened a pull request:

    https://github.com/apache/spark/pull/19532

    [CORE]stage api modify the description format, add version api, modify the duration real-time calculation

    
    ## What changes were proposed in this pull request?
    
    
    stage api modify the description format
    <td>A list of all stages for a given application.</td>
    <br><code>?status=[active|complete|pending|failed]</code> list only stages in the state.
    content should be included in <td> </ td>
    add version api doc '/api/v1/version'
    modify the duration real-time calculation in running appcations
    
    ## How was this patch tested?
    manual tests
    
    Please review http://spark.apache.org/contributing.html before opening a pull request.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/guoxiaolongzte/spark SPARK-22311

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/19532.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #19532
    
----
commit 8f53eceb9ed3c33388cef09f628dfb7e4f6de70d
Author: guoxiaolong <gu...@zte.com.cn>
Date:   2017-10-19T03:15:13Z

    [CORE]stage api modify the description format, add version api, modify the duration real-time calculation

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    **[Test build #83377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83377/testReport)** for PR 19532 at commit [`9670d6f`](https://github.com/apache/spark/commit/9670d6f8b3fed58f556b9050814f7c61bc5d65be).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]stage api modify the description format, add versi...

Posted by jiangxb1987 <gi...@git.apache.org>.

Github user jiangxb1987 commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Please update the title to:
    ```
    Modify the duration real-time calculation and update the API doc
    ```


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83213/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    **[Test build #83637 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83637/testReport)** for PR 19532 at commit [`9670d6f`](https://github.com/apache/spark/commit/9670d6f8b3fed58f556b9050814f7c61bc5d65be).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by guoxiaolongzte <gi...@git.apache.org>.

Github user guoxiaolongzte commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    @jiangxb1987 I modified it.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #19532: [CORE]Modify the duration real-time calculation a...

Posted by ajbozarth <gi...@git.apache.org>.

Github user ajbozarth commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19532#discussion_r147795197
  
    --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
    @@ -120,7 +120,7 @@ private[spark] class SparkUI private (
             attemptId = None,
             startTime = new Date(startTime),
             endTime = new Date(-1),
    -        duration = 0,
    +        duration = System.currentTimeMillis() - startTime,
    --- End diff --
    
    I'm on the fence on this change, I would tend to agree with @srowen but as @cloud-fan said, the consistency with the UI would make sense. If we did keep this change I thing removing the redundant code would also make sense.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    **[Test build #83213 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83213/testReport)** for PR 19532 at commit [`8f53ece`](https://github.com/apache/spark/commit/8f53eceb9ed3c33388cef09f628dfb7e4f6de70d).
     * This patch **fails build dependency tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by guoxiaolongzte <gi...@git.apache.org>.

Github user guoxiaolongzte commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #19532: [DOC]update the API doc and modify the stage API ...

Posted by asfgit <gi...@git.apache.org>.

Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/19532


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by guoxiaolongzte <gi...@git.apache.org>.

Github user guoxiaolongzte commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    @cloud-fan
    Help merge the code.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #19532: [CORE]Modify the duration real-time calculation a...

Posted by guoxiaolongzte <gi...@git.apache.org>.

Github user guoxiaolongzte commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19532#discussion_r147677958
  
    --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
    @@ -120,7 +120,7 @@ private[spark] class SparkUI private (
             attemptId = None,
             startTime = new Date(startTime),
             endTime = new Date(-1),
    -        duration = 0,
    +        duration = System.currentTimeMillis() - startTime,
    --- End diff --
    
    No effect.
    In master page, ‘duration ’is calculated using the duration attribute of the ApplicationInfo object.
    ![1](https://user-images.githubusercontent.com/26266482/32168880-2925daf4-bda9-11e7-8e89-925d5d29bf2c.png)
    
    ![2](https://user-images.githubusercontent.com/26266482/32168885-2e6c9aca-bda9-11e7-9f77-085c4721ead3.png)
    
    And ApplicationAttemptInfo object does not matter.
    
    
     


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83214/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by cloud-fan <gi...@git.apache.org>.

Github user cloud-fan commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    thanks, merging to master!


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by cloud-fan <gi...@git.apache.org>.

Github user cloud-fan commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    ok to test


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by guoxiaolongzte <gi...@git.apache.org>.

Github user guoxiaolongzte commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    I have updated the title and description.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by jiangxb1987 <gi...@git.apache.org>.

Github user jiangxb1987 commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83386/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by guoxiaolongzte <gi...@git.apache.org>.

Github user guoxiaolongzte commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Thank you for your review comments, I have to restore the code, not running in the code calculation. Now only keep the document changes. Please review again.
    @srowen @jiangxb1987 @cloud-fan @ajbozarth 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    **[Test build #83214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83214/testReport)** for PR 19532 at commit [`8f53ece`](https://github.com/apache/spark/commit/8f53eceb9ed3c33388cef09f628dfb7e4f6de70d).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by guoxiaolongzte <gi...@git.apache.org>.

Github user guoxiaolongzte commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    @jiangxb1987  @srowen 
    Help review the code.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by cloud-fan <gi...@git.apache.org>.

Github user cloud-fan commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    **[Test build #83377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83377/testReport)** for PR 19532 at commit [`9670d6f`](https://github.com/apache/spark/commit/9670d6f8b3fed58f556b9050814f7c61bc5d65be).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83377/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #19532: [CORE]Modify the duration real-time calculation a...

Posted by cloud-fan <gi...@git.apache.org>.

Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19532#discussion_r147673834
  
    --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
    @@ -120,7 +120,7 @@ private[spark] class SparkUI private (
             attemptId = None,
             startTime = new Date(startTime),
             endTime = new Date(-1),
    -        duration = 0,
    +        duration = System.currentTimeMillis() - startTime,
    --- End diff --
    
    does this have an effect on the UI?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by cloud-fan <gi...@git.apache.org>.

Github user cloud-fan commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]Modify the duration real-time calculation and upda...

Posted by ajbozarth <gi...@git.apache.org>.

Github user ajbozarth commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Thanks, if you could update the title and description to match as well.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [CORE]stage api modify the description format, add versi...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    The other change is OK though


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83411/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19532: [DOC]update the API doc and modify the stage API descrip...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19532
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org