You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by pgandhi999 <gi...@git.apache.org> on 2017/09/12 14:43:51 UTC

[GitHub] spark pull request #19207: [SPARK-21809] : Change Stage Page to use datatabl...

GitHub user pgandhi999 opened a pull request:

    https://github.com/apache/spark/pull/19207

    [SPARK-21809] : Change Stage Page to use datatables to support sorting columns and searching

    Support column sort and search for Stage Server using jQuery DataTable and REST API. Before this commit, the Stage page was generated hard-coded HTML and can not support search, also, the sorting was disabled if there is any application that has more than one attempt. Supporting search and sort (over all applications rather than the 20 entries in the current page) in any case will greatly improve the user experience.
    Created the stagespage-template.html for displaying application information in datables. Added REST api endpoint and javascript code to fetch data from the endpoint and display it on the data table.
    
    ## How was this patch tested?
    I have attached the screenshots of the Stage Page UI before and after the fix.
    Before:
    <img width="1419" alt="screen shot-before" src="https://user-images.githubusercontent.com/22228190/30331985-d773d9ac-979e-11e7-8920-5d11fdf8766a.png">
    
    After:
    <img width="1423" alt="screen shot- after" src="https://user-images.githubusercontent.com/22228190/30331998-dd22a1d0-979e-11e7-860e-3694e45cd782.png">
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/pgandhi999/spark master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/19207.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #19207
    
----
commit 172fc20898896058b7288360eb5292ed9df9d79c
Author: pgandhi <pg...@yahoo-inc.com>
Date:   2017-07-21T21:00:22Z

    [SPARK-21503]: Fixed the issue
    
    Added the case ExecutorLostFailure which was previously not there, thus, the default case would be executed in which case, task would be marked as completed.

commit 81422e0f634c0f06eb2ea29fba4281176a1ab528
Author: pgandhi <pg...@yahoo-inc.com>
Date:   2017-07-25T14:54:41Z

    [SPARK-21503][UI]: Adding changes as per comments

commit 55c6c37d09b41ae6914edb5d067e7f2c252ac92a
Author: pgandhi999 <pa...@gmail.com>
Date:   2017-07-26T21:26:27Z

    Merge pull request #1 from apache/master
    
    Apache Spark Pull Request - July 26, 2017

commit f454c8933e07967548095e068063bd313ae4845c
Author: pgandhi <pg...@yahoo-inc.com>
Date:   2017-07-26T21:41:16Z

    [SPARK-21541]: Spark Logs show incorrect job status for a job that does not create SparkContext
    
    Added a flag to check whether user has initialized Spark Context. If it is true, then we let Application Master unregister with Resource Manager else we do not.

commit 6b7d5c6e2565c7c4dd97f31fe404c59e73c7474c
Author: pgandhi <pg...@yahoo-inc.com>
Date:   2017-07-26T21:58:27Z

    Revert "[SPARK-21541]: Spark Logs show incorrect job status for a job that does not create SparkContext"
    
    This reverts commit f454c8933e07967548095e068063bd313ae4845c.
    
    "Merged another issue to this one by mistake"

commit bc4166490d2ff68898c00fae4c1ca1b8abe1e795
Author: pgandhi999 <pa...@gmail.com>
Date:   2017-07-28T15:24:55Z

    Merge pull request #2 from apache/master
    
    Spark - July 28, 2017

commit e46126fe0f3d8d6f92f7f51c30d8c2154bddc126
Author: pgandhi <pg...@yahoo-inc.com>
Date:   2017-07-28T16:08:08Z

    [SPARK-21503]- Making Changes as per comments
    
    [SPARK-21503]- Making Changes as per comments: Removed match case statement and replaced it with an if clause.

commit 9b3cebc6b65d2da835f02efaa27015cfd1b0ccae
Author: pgandhi999 <pa...@gmail.com>
Date:   2017-08-01T13:58:12Z

    Merge pull request #4 from apache/master
    
    Spark - August 1, 2017

commit 7f03341093c843086920e8218463b5d2ba6e37d2
Author: pgandhi <pg...@yahoo-inc.com>
Date:   2017-08-01T15:52:13Z

    [SPARK-21503]: Reverting Unit Test Code
    
    [SPARK-21503]: Reverting Unit Test Code - Not needed.

commit 2d01cab45ae269db9044815970dd008c851a46cc
Author: pgandhi999 <pa...@gmail.com>
Date:   2017-08-24T21:59:52Z

    Merge pull request #5 from apache/master
    
    SPARK - August 24, 2017

commit eaf63e6bd4dddc726cf57fda080b9b5d6341e2f8
Author: pgandhi <pg...@yahoo-inc.com>
Date:   2017-08-24T22:03:29Z

    [SPARK-21798]: No config to replace deprecated SPARK_CLASSPATH config for launching daemons like History Server
    
    Adding new env variable SPARK_DAEMON_CLASSPATH to set classpath for launching daemons. Tested and verified for History Server and Standalone Mode.

commit e421a03acbd410a835cf3117fe6592523dc649b5
Author: pgandhi <pg...@yahoo-inc.com>
Date:   2017-08-25T16:13:47Z

    [SPARK-21798]: No config to replace deprecated SPARK_CLASSPATH config for launching daemons like History Server
    
    Reverted the previous code change and added the environment variable SPARK_DAEMON_CLASSPATH only for launching daemon processes.

commit 6dfb0b0a0ff571850835386664e43f4788d0f046
Author: Parth Gandhi <pa...@gmail.com>
Date:   2017-09-11T18:15:23Z

    Merge pull request #6 from apache/master
    
    Spark - September 11, 2017

commit d95d69b110f27e409b9185d694cde13a472762c2
Author: pgandhi <pg...@yahoo-inc.com>
Date:   2017-09-12T14:06:22Z

    [SPARK-21809] : Change Stage Page to use datatables to support sorting columns and searching
    
    Converted static html tables to datatables on stage page

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19207
  
    **[Test build #81784 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81784/testReport)** for PR 19207 at commit [`f349a39`](https://github.com/apache/spark/commit/f349a392376f87e0bf24b2cd6d64d751fe62ed17).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19207
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81681/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19207
  
    **[Test build #81683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81683/testReport)** for PR 19207 at commit [`20e04fa`](https://github.com/apache/spark/commit/20e04fa5e45556b7945203e332e6c4bb2f719e3a).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19207
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81783/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19207
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19207
  
    **[Test build #81736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81736/testReport)** for PR 19207 at commit [`de21671`](https://github.com/apache/spark/commit/de2167186d5f4c89fc00b7b95a79acfc234b3da3).
     * This patch **fails MiMa tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

Posted by pgandhi999 <gi...@git.apache.org>.

Github user pgandhi999 commented on the issue:

    https://github.com/apache/spark/pull/19207
  
    The error logs for test build #81683 state that method this(Long,Int,Int,Long,Long,Long,Long,Long,Long)Unit in class org.apache.spark.status.api.v1.ExecutorStageSummary does not have a correspondent in current version. All I have done is add new fields in the api ExecutorStageSummary and have not modified any existing ones. It should be fine but please let me know if it is not.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

Posted by tgravescs <gi...@git.apache.org>.

Github user tgravescs commented on the issue:

    https://github.com/apache/spark/pull/19207
  
    ok to test


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19207
  
    Can one of the admins verify this patch?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org