[jira] [Created] (SPARK-25504) [Spark Job History]If the number of tasks exceeds spark.ui.retainedTask value, then stage page of UI should display upto the value set in the retainedTask.

["ABHISHEK KUMAR GUPTA (JIRA)" <ji...@apache.org>]

ABHISHEK KUMAR GUPTA created SPARK-25504:
--------------------------------------------

             Summary: [Spark Job History]If the number of tasks exceeds spark.ui.retainedTask value, then stage page of UI should display upto the value set in the retainedTask.
                 Key: SPARK-25504
                 URL: https://issues.apache.org/jira/browse/SPARK-25504
             Project: Spark
          Issue Type: Bug
          Components: Web UI
    Affects Versions: 2.3.1
            Reporter: ABHISHEK KUMAR GUPTA


*Steps*
 1. spark.ui.retainedTask =60000( spark-default.conf of Job History)
 2. Run a job for creating 100 task as below from Spark-shell
 val rdd = sc.parallelize(1 to 49000, 49000)
 rdd.count
 3. Launch Job History Page
 4. Click on corresponding spark-shell Application ID link in Job History and Launch Job Page for the application
 5. Click on the Job ID description and Launch Stage Page
 6. Click on the Description of Stage ID to Launch Stage Page of specific Stage ID

*Atcual Result*
 Showing Total number of task above Pagination Panel as *Tasks(49000)*

It is correct as number of task is 49000 created.

Now Execute as below

1. Run a job for creating 100010 task as below from Spark-shell
 val rdd = sc.parallelize(1 to 100100, 100100))
 rdd.count
 2. Launch Job History Page
 3. Click on corresponding spark-shell Application ID link in Job History and Launch Job Page for the application
 4. Click on the Job ID description and Launch Stage Page
 5. Click on the Description of Stage ID to Launch Stage Page of specific Stage ID

*Atcual Result*
 Showing Total number of task above Pagination Panel as *Tasks(58100, showing 100100)*

*Expected:*
 spark.ui.retainedTask =60000
 So it should show as *Tasks( 60000, Showing 100100)* as spark.ui.retainedTask =60000



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org