[jira] [Commented] (BIGTOP-1065) Stress Tests

["jay vyas (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13916617#comment-13916617 ] 

jay vyas commented on BIGTOP-1065:
----------------------------------

The activity on BIGTOP-1198 seems to have supplanted this JIRA, and is redundant.  lets close this one.  Yay for stress tests !

> Stress Tests 
> -------------
>
>                 Key: BIGTOP-1065
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-1065
>             Project: Bigtop
>          Issue Type: Test
>            Reporter: jay vyas
>            Priority: Minor
>
> There are issues in the past where the JobTracker has failures because of so called "memory-leaks" , or infinitely growing objects, unclean tmp/ folders from jobs, etc.  
> Thus, the proposal here is a lighteweight job which can be run rapidly, several 1000 times, which confirms that jobtracker state does not grow out of bounds or infinitely with respect to number of tasks/jobs run/submitted. 
> IMPLEMENTATION PROPOSAL: 
> Some simple starts would be to : 
> - run word count, or a the sleep job 100 or 1000 times or 10,000 times.  
> - create and delete the same file over and over again several thousand times to see if filesystem consistency is maintained 
> To start, I'd like to add all these tests in a single module , under test-executions/stress/.  Then later we could shard it out in another way.
> UPDATE: 
> As per comments below, just noting that although phrased in terms of "JobTracker", the spirit of this ticket is to be applicable in both mr1 and mr2, since in either case, the purpose is to test the impact that several 100/1000 mapreduce job runs has over time and confirm that tmp dirs, objects in memory, etc are all managed and lifecycled properly .



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)