You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Ahmed Hussein (Jira)" <ji...@apache.org> on 2020/01/24 21:47:00 UTC
[jira] [Created] (MAPREDUCE-7262) MRApp helpers blocks for long
intervals
Ahmed Hussein created MAPREDUCE-7262:
----------------------------------------
Summary: MRApp helpers blocks for long intervals
Key: MAPREDUCE-7262
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7262
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mr-am
Reporter: Ahmed Hussein
Assignee: Ahmed Hussein
MRApp has a set of methods used as helpers in test cases such as: {{waitForInternalState(TA)}}, {{waitForState(TA)}}, {{waitForState(Job)}}..etc
When the condition fails, the thread sleeps for a minimum of 500ms before rechecking the new state of the Job/TA.
Example:
{code:java}
public void waitForState(Task task, TaskState finalState) throws Exception {
int timeoutSecs = 0;
TaskReport report = task.getReport();
while (!finalState.equals(report.getTaskState()) &&
timeoutSecs++ < 20) {
System.out.println("Task State for " + task.getID() + " is : "
+ report.getTaskState() + " Waiting for state : " + finalState
+ " progress : " + report.getProgress());
report = task.getReport();
Thread.sleep(500);
}
System.out.println("Task State is : " + report.getTaskState());
Assert.assertEquals("Task state is not correct (timedout)", finalState,
report.getTaskState());
}
{code}
I suggest to reduce the interval 500 to 50, while incrementing the number of retries to 200. this will potentially make the test cases run faster. Also, the {{System.out}} calls need to be removed because they are not adding information dumping the current state on every iteration.
A tentative list of Junits affected by the change:
{code:bash}
Method
waitForInternalState(JobImpl, JobStateInternal)
Found usages (12 usages found)
org.apache.hadoop.mapreduce.v2.app (10 usages found)
TestJobEndNotifier (3 usages found)
testNotificationOnLastRetry(boolean) (1 usage found)
214 app.waitForInternalState(job, JobStateInternal.SUCCEEDED);
testAbsentNotificationOnNotLastRetryUnregistrationFailure() (1 usage found)
256 app.waitForInternalState(job, JobStateInternal.REBOOT);
testNotificationOnLastRetryUnregistrationFailure() (1 usage found)
289 app.waitForInternalState(job, JobStateInternal.REBOOT);
TestKill (5 usages found)
testKillJob() (1 usage found)
70 app.waitForInternalState((JobImpl) job, JobStateInternal.RUNNING);
testKillTask() (1 usage found)
108 app.waitForInternalState((JobImpl) job, JobStateInternal.RUNNING);
testKillTaskWait() (1 usage found)
219 app.waitForInternalState((JobImpl) job, JobStateInternal.KILLED);
testKillTaskWaitKillJobAfterTA_DONE() (1 usage found)
266 app.waitForInternalState((JobImpl)job, JobStateInternal.KILLED);
testKillTaskWaitKillJobBeforeTA_DONE() (1 usage found)
316 app.waitForInternalState((JobImpl)job, JobStateInternal.KILLED);
TestMRApp (2 usages found)
testJobSuccess() (1 usage found)
494 app.waitForInternalState(job, JobStateInternal.SUCCEEDED);
testJobRebootOnLastRetryOnUnregistrationFailure() (1 usage found)
542 app.waitForInternalState((JobImpl) job, JobStateInternal.REBOOT);
org.apache.hadoop.mapreduce.v2.app.rm (2 usages found)
TestRMContainerAllocator (2 usages found)
testReportedAppProgress() (1 usage found)
1050 mrApp.waitForInternalState((JobImpl) job, JobStateInternal.RUNNING);
testReportedAppProgressWithOnlyMaps() (1 usage found)
1202 mrApp.waitForInternalState((JobImpl)job, JobStateInternal.RUNNING);
--------------------------------------------------------------------------------------
Method
waitForState(TaskAttempt, TaskAttemptState)
Found usages (72 usages found)
org.apache.hadoop.mapreduce.v2 (2 usages found)
TestSpeculativeExecutionWithMRApp (2 usages found)
testSpeculateSuccessfulWithoutUpdateEvents() (1 usage found)
212 app.waitForState(taskAttempt.getValue(), TaskAttemptState.SUCCEEDED);
testSpeculateSuccessfulWithUpdateEvents() (1 usage found)
275 app.waitForState(taskAttempt.getValue(), TaskAttemptState.SUCCEEDED);
org.apache.hadoop.mapreduce.v2.app (67 usages found)
TestAMInfos (1 usage found)
testAMInfosWithoutRecoveryEnabled() (1 usage found)
58 app.waitForState(taskAttempt, TaskAttemptState.RUNNING);
TestFetchFailure (11 usages found)
testFetchFailure() (3 usages found)
77 app.waitForState(mapAttempt1, TaskAttemptState.RUNNING);
109 app.waitForState(reduceAttempt, TaskAttemptState.RUNNING);
130 app.waitForState(mapAttempt2, TaskAttemptState.RUNNING);
testFetchFailureWithRecovery() (3 usages found)
209 app.waitForState(mapAttempt1, TaskAttemptState.RUNNING);
230 app.waitForState(reduceAttempt, TaskAttemptState.RUNNING);
262 app.waitForState(mapAttempt1, TaskAttemptState.RUNNING);
testFetchFailureMultipleReduces() (5 usages found)
304 app.waitForState(mapAttempt1, TaskAttemptState.RUNNING);
327 app.waitForState(reduceAttempt, TaskAttemptState.RUNNING);
333 app.waitForState(reduceAttempt2, TaskAttemptState.RUNNING);
338 app.waitForState(reduceAttempt3, TaskAttemptState.RUNNING);
372 app.waitForState(mapAttempt2, TaskAttemptState.RUNNING);
TestKill (6 usages found)
testKillTaskWait() (2 usages found)
204 app.waitForState(mapAttempt, TaskAttemptState.RUNNING);
206 app.waitForState(reduceAttempt, TaskAttemptState.RUNNING);
testKillTaskWaitKillJobAfterTA_DONE() (2 usages found)
242 app.waitForState(mapAttempt, TaskAttemptState.RUNNING);
244 app.waitForState(reduceAttempt, TaskAttemptState.RUNNING);
testKillTaskWaitKillJobBeforeTA_DONE() (2 usages found)
290 app.waitForState(mapAttempt, TaskAttemptState.RUNNING);
292 app.waitForState(reduceAttempt, TaskAttemptState.RUNNING);
TestMRApp (7 usages found)
testCommitPending() (3 usages found)
114 app.waitForState(attempt, TaskAttemptState.RUNNING);
123 app.waitForState(attempt, TaskAttemptState.COMMIT_PENDING);
132 app.waitForState(attempt, TaskAttemptState.COMMIT_PENDING);
testCompletedMapsForReduceSlowstart() (2 usages found)
169 app.waitForState(task1Attempt, TaskAttemptState.RUNNING);
170 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
testUpdatedNodes() (2 usages found)
287 app.waitForState(task1Attempt, TaskAttemptState.KILLED);
288 app.waitForState(task2Attempt, TaskAttemptState.KILLED);
TestMRClientService (2 usages found)
test() (1 usage found)
90 app.waitForState(attempt, TaskAttemptState.RUNNING);
testViewAclOnlyCannotModify() (1 usage found)
215 app.waitForState(attempt, TaskAttemptState.RUNNING);
TestRecovery (40 usages found)
testCrashed() (8 usages found)
165 app.waitForState(task1Attempt1, TaskAttemptState.RUNNING);
166 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
176 app.waitForState(task1Attempt1, TaskAttemptState.FAILED);
197 app.waitForState(task1Attempt2, TaskAttemptState.FAILED);
210 app.waitForState(task1Attempt3, TaskAttemptState.RUNNING);
218 app.waitForState(task1Attempt3, TaskAttemptState.KILLED);
232 app.waitForState(task1Attempt4, TaskAttemptState.RUNNING);
278 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
testCrashOfMapsOnlyJob() (4 usages found)
382 app.waitForState(task1Attempt, TaskAttemptState.RUNNING);
383 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
384 app.waitForState(task3Attempt, TaskAttemptState.RUNNING);
435 app.waitForState(task3Attempt, TaskAttemptState.RUNNING);
testRecoverySuccessUsingCustomOutputCommitter() (4 usages found)
527 app.waitForState(task1Attempt, TaskAttemptState.RUNNING);
528 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
529 app.waitForState(task3Attempt, TaskAttemptState.RUNNING);
581 app.waitForState(task3Attempt, TaskAttemptState.RUNNING);
testRecoveryWithSpillEncryption() (3 usages found)
621 app.waitForState(mapAttempt, TaskAttemptState.RUNNING);
647 app.waitForState(mapAttempt, TaskAttemptState.RUNNING);
654 app.waitForState(redAttempt, TaskAttemptState.RUNNING);
testRecoveryFailsUsingCustomOutputCommitter() (4 usages found)
706 app.waitForState(task1Attempt, TaskAttemptState.RUNNING);
707 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
708 app.waitForState(task3Attempt, TaskAttemptState.RUNNING);
760 app.waitForState(task3Attempt, TaskAttemptState.RUNNING);
testMultipleCrashes() (3 usages found)
821 app.waitForState(task1Attempt1, TaskAttemptState.RUNNING);
822 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
869 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
testOutputRecovery() (2 usages found)
946 app.waitForState(task1Attempt1, TaskAttemptState.RUNNING);
1013 app.waitForState(reduce2Attempt, TaskAttemptState.RUNNING);
testOutputRecoveryMapsOnly() (2 usages found)
1123 app.waitForState(task1Attempt1, TaskAttemptState.RUNNING);
1177 app.waitForState(task2Attempt1, TaskAttemptState.RUNNING);
testRecoveryWithOldCommiter() (2 usages found)
1237 app.waitForState(task1Attempt1, TaskAttemptState.RUNNING);
1304 app.waitForState(reduce2Attempt, TaskAttemptState.RUNNING);
testSpeculative() (5 usages found)
1382 app.waitForState(task1Attempt1, TaskAttemptState.RUNNING);
1383 app.waitForState(task1Attempt2, TaskAttemptState.RUNNING);
1384 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
1394 app.waitForState(task1Attempt1, TaskAttemptState.SUCCEEDED);
1432 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
testRecoveryWithoutShuffleSecret() (3 usages found)
1511 app.waitForState(task1Attempt, TaskAttemptState.RUNNING);
1512 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
1555 app.waitForState(task2Attempt, TaskAttemptState.RUNNING);
org.apache.hadoop.mapreduce.v2.app.job.impl (3 usages found)
TestTaskAttempt (3 usages found)
verifyMillisCounters(Resource, int) (2 usages found)
370 app.waitForState(mta, TaskAttemptState.RUNNING);
371 app.waitForState(rta, TaskAttemptState.RUNNING);
testTaskAttemptAssignedKilledHistory(FailingAttemptsDuringAssignedMRApp) (1 usage found)
488 app.waitForState(attempt, TaskAttemptState.KILLED);
-------------------------------------------
Method
waitForState(Task, TaskState)
Found usages (155 usages found)
org.apache.hadoop.mapreduce.v2 (2 usages found)
TestSpeculativeExecutionWithMRApp (2 usages found)
testSpeculateSuccessfulWithoutUpdateEvents() (1 usage found)
179 app.waitForState(taskIter.next(), TaskState.RUNNING);
testSpeculateSuccessfulWithUpdateEvents() (1 usage found)
241 app.waitForState(taskIter.next(), TaskState.RUNNING);
org.apache.hadoop.mapreduce.v2.app (145 usages found)
TestAMInfos (1 usage found)
testAMInfosWithoutRecoveryEnabled() (1 usage found)
56 app.waitForState(mapTask, TaskState.RUNNING);
TestFail (1 usage found)
testTaskFailWithUnusedContainer() (1 usage found)
192 app.waitForState(task, TaskState.SCHEDULED);
TestFetchFailure (18 usages found)
testFetchFailure() (5 usages found)
75 app.waitForState(mapTask, TaskState.RUNNING);
85 app.waitForState(mapTask, TaskState.SUCCEEDED);
106 app.waitForState(reduceTask, TaskState.RUNNING);
117 app.waitForState(mapTask, TaskState.RUNNING);
137 app.waitForState(mapTask, TaskState.SUCCEEDED);
testFetchFailureWithRecovery() (6 usages found)
207 app.waitForState(mapTask, TaskState.RUNNING);
217 app.waitForState(mapTask, TaskState.SUCCEEDED);
227 app.waitForState(reduceTask, TaskState.RUNNING);
238 app.waitForState(mapTask, TaskState.RUNNING);
260 app.waitForState(mapTask, TaskState.RUNNING);
270 app.waitForState(mapTask, TaskState.SUCCEEDED);
testFetchFailureMultipleReduces() (7 usages found)
302 app.waitForState(mapTask, TaskState.RUNNING);
312 app.waitForState(mapTask, TaskState.SUCCEEDED);
322 app.waitForState(reduceTask, TaskState.RUNNING);
323 app.waitForState(reduceTask2, TaskState.RUNNING);
324 app.waitForState(reduceTask3, TaskState.RUNNING);
354 app.waitForState(mapTask, TaskState.RUNNING);
379 app.waitForState(mapTask, TaskState.SUCCEEDED);
TestKill (9 usages found)
testKillTaskWait() (3 usages found)
201 app.waitForState(mapTask, TaskState.RUNNING);
202 app.waitForState(reduceTask, TaskState.RUNNING);
213 app.waitForState(mapTask, TaskState.SUCCEEDED);
testKillTaskWaitKillJobAfterTA_DONE() (2 usages found)
239 app.waitForState(mapTask, TaskState.RUNNING);
240 app.waitForState(reduceTask, TaskState.RUNNING);
testKillTaskWaitKillJobBeforeTA_DONE() (2 usages found)
287 app.waitForState(mapTask, TaskState.RUNNING);
288 app.waitForState(reduceTask, TaskState.RUNNING);
testKillTaskAttempt() (2 usages found)
380 app.waitForState(task1, TaskState.SCHEDULED);
381 app.waitForState(task2, TaskState.SCHEDULED);
TestMRApp (24 usages found)
testCommitPending() (1 usage found)
112 app.waitForState(task, TaskState.RUNNING);
testCompletedMapsForReduceSlowstart() (4 usages found)
161 app.waitForState(mapTask1, TaskState.RUNNING);
162 app.waitForState(mapTask2, TaskState.RUNNING);
183 app.waitForState(mapTask1, TaskState.SUCCEEDED);
187 app.waitForState(reduceTask, TaskState.RUNNING);
testUpdatedNodes() (16 usages found)
233 app.waitForState(mapTask1, TaskState.RUNNING);
234 app.waitForState(mapTask2, TaskState.RUNNING);
257 app.waitForState(mapTask1, TaskState.SUCCEEDED);
258 app.waitForState(mapTask2, TaskState.SUCCEEDED);
310 app.waitForState(mapTask1, TaskState.RUNNING);
311 app.waitForState(mapTask2, TaskState.RUNNING);
325 app.waitForState(mapTask1, TaskState.SUCCEEDED);
326 app.waitForState(mapTask2, TaskState.RUNNING);
362 app.waitForState(mapTask1, TaskState.SUCCEEDED);
363 app.waitForState(mapTask2, TaskState.RUNNING);
385 app.waitForState(mapTask2, TaskState.SUCCEEDED);
400 app.waitForState(reduceTask1, TaskState.RUNNING);
401 app.waitForState(reduceTask2, TaskState.RUNNING);
410 app.waitForState(reduceTask1, TaskState.SUCCEEDED);
416 app.waitForState(reduceTask1, TaskState.SUCCEEDED);
425 app.waitForState(reduceTask2, TaskState.SUCCEEDED);
testJobError() (1 usage found)
478 app.waitForState(task, TaskState.RUNNING);
testJobRebootNotLastRetryOnUnregistrationFailure() (1 usage found)
511 app.waitForState(task, TaskState.RUNNING);
testJobRebootOnLastRetryOnUnregistrationFailure() (1 usage found)
536 app.waitForState(task, TaskState.RUNNING);
TestMRClientService (2 usages found)
test() (1 usage found)
88 app.waitForState(task, TaskState.RUNNING);
testViewAclOnlyCannotModify() (1 usage found)
213 app.waitForState(task, TaskState.RUNNING);
TestRecovery (90 usages found)
testCrashed() (8 usages found)
157 app.waitForState(mapTask1, TaskState.RUNNING);
158 app.waitForState(mapTask2, TaskState.RUNNING);
168 app.waitForState(reduceTask, TaskState.RUNNING);
243 app.waitForState(mapTask1, TaskState.SUCCEEDED);
271 app.waitForState(mapTask1, TaskState.SUCCEEDED);
273 app.waitForState(mapTask2, TaskState.RUNNING);
287 app.waitForState(mapTask2, TaskState.SUCCEEDED);
290 app.waitForState(reduceTask, TaskState.RUNNING);
testCrashOfMapsOnlyJob() (9 usages found)
369 app.waitForState(mapTask1, TaskState.RUNNING);
370 app.waitForState(mapTask2, TaskState.RUNNING);
371 app.waitForState(mapTask3, TaskState.RUNNING);
399 app.waitForState(mapTask1, TaskState.SUCCEEDED);
400 app.waitForState(mapTask2, TaskState.SUCCEEDED);
427 app.waitForState(mapTask1, TaskState.SUCCEEDED);
428 app.waitForState(mapTask2, TaskState.SUCCEEDED);
430 app.waitForState(mapTask3, TaskState.RUNNING);
446 app.waitForState(mapTask3, TaskState.SUCCEEDED);
testRecoverySuccessUsingCustomOutputCommitter() (9 usages found)
514 app.waitForState(mapTask1, TaskState.RUNNING);
515 app.waitForState(mapTask2, TaskState.RUNNING);
516 app.waitForState(mapTask3, TaskState.RUNNING);
544 app.waitForState(mapTask1, TaskState.SUCCEEDED);
545 app.waitForState(mapTask2, TaskState.SUCCEEDED);
573 app.waitForState(mapTask1, TaskState.SUCCEEDED);
574 app.waitForState(mapTask2, TaskState.SUCCEEDED);
576 app.waitForState(mapTask3, TaskState.RUNNING);
591 app.waitForState(mapTask3, TaskState.SUCCEEDED);
testRecoveryWithSpillEncryption() (5 usages found)
619 app.waitForState(mapper, TaskState.RUNNING);
624 app.waitForState(mapper, TaskState.SUCCEEDED);
645 app.waitForState(mapper, TaskState.RUNNING);
650 app.waitForState(mapper, TaskState.SUCCEEDED);
657 app.waitForState(reducer, TaskState.SUCCEEDED);
testRecoveryFailsUsingCustomOutputCommitter() (9 usages found)
693 app.waitForState(mapTask1, TaskState.RUNNING);
694 app.waitForState(mapTask2, TaskState.RUNNING);
695 app.waitForState(mapTask3, TaskState.RUNNING);
723 app.waitForState(mapTask1, TaskState.SUCCEEDED);
724 app.waitForState(mapTask2, TaskState.SUCCEEDED);
752 app.waitForState(mapTask1, TaskState.RUNNING);
753 app.waitForState(mapTask2, TaskState.RUNNING);
755 app.waitForState(mapTask3, TaskState.RUNNING);
784 app.waitForState(mapTask3, TaskState.SUCCEEDED);
testMultipleCrashes() (9 usages found)
813 app.waitForState(mapTask1, TaskState.RUNNING);
814 app.waitForState(mapTask2, TaskState.RUNNING);
835 app.waitForState(mapTask1, TaskState.SUCCEEDED);
862 app.waitForState(mapTask1, TaskState.SUCCEEDED);
864 app.waitForState(mapTask2, TaskState.RUNNING);
878 app.waitForState(mapTask2, TaskState.SUCCEEDED);
905 app.waitForState(mapTask1, TaskState.SUCCEEDED);
906 app.waitForState(mapTask2, TaskState.SUCCEEDED);
909 app.waitForState(reduceTask, TaskState.RUNNING);
testOutputRecovery() (8 usages found)
939 app.waitForState(mapTask1, TaskState.RUNNING);
955 app.waitForState(mapTask1, TaskState.SUCCEEDED);
960 app.waitForState(reduceTask1, TaskState.RUNNING);
973 app.waitForState(reduceTask1, TaskState.SUCCEEDED);
998 app.waitForState(mapTask1, TaskState.SUCCEEDED);
1005 app.waitForState(reduceTask1, TaskState.SUCCEEDED);
1007 app.waitForState(reduceTask2, TaskState.RUNNING);
1022 app.waitForState(reduceTask2, TaskState.SUCCEEDED);
testOutputRecoveryMapsOnly() (7 usages found)
1116 app.waitForState(mapTask1, TaskState.RUNNING);
1136 app.waitForState(mapTask1, TaskState.SUCCEEDED);
1164 app.waitForState(mapTask1, TaskState.SUCCEEDED);
1170 app.waitForState(mapTask2, TaskState.RUNNING);
1186 app.waitForState(mapTask2, TaskState.SUCCEEDED);
1191 app.waitForState(reduceTask1, TaskState.RUNNING);
1204 app.waitForState(reduceTask1, TaskState.SUCCEEDED);
testRecoveryWithOldCommiter() (8 usages found)
1230 app.waitForState(mapTask1, TaskState.RUNNING);
1246 app.waitForState(mapTask1, TaskState.SUCCEEDED);
1251 app.waitForState(reduceTask1, TaskState.RUNNING);
1264 app.waitForState(reduceTask1, TaskState.SUCCEEDED);
1289 app.waitForState(mapTask1, TaskState.SUCCEEDED);
1296 app.waitForState(reduceTask1, TaskState.SUCCEEDED);
1298 app.waitForState(reduceTask2, TaskState.RUNNING);
1313 app.waitForState(reduceTask2, TaskState.SUCCEEDED);
testSpeculative() (8 usages found)
1352 app.waitForState(mapTask1, TaskState.RUNNING);
1353 app.waitForState(mapTask2, TaskState.RUNNING);
1386 app.waitForState(reduceTask, TaskState.RUNNING);
1397 app.waitForState(mapTask1, TaskState.SUCCEEDED);
1425 app.waitForState(mapTask1, TaskState.SUCCEEDED);
1427 app.waitForState(mapTask2, TaskState.RUNNING);
1441 app.waitForState(mapTask2, TaskState.SUCCEEDED);
1444 app.waitForState(reduceTask, TaskState.RUNNING);
testRecoveryWithoutShuffleSecret() (10 usages found)
1503 app.waitForState(mapTask1, TaskState.RUNNING);
1504 app.waitForState(mapTask2, TaskState.RUNNING);
1514 app.waitForState(reduceTask, TaskState.RUNNING);
1523 app.waitForState(mapTask1, TaskState.SUCCEEDED);
1549 app.waitForState(mapTask1, TaskState.RUNNING);
1550 app.waitForState(mapTask2, TaskState.RUNNING);
1564 app.waitForState(mapTask2, TaskState.SUCCEEDED);
1567 app.waitForState(mapTask1, TaskState.RUNNING);
1576 app.waitForState(mapTask1, TaskState.SUCCEEDED);
1579 app.waitForState(reduceTask, TaskState.RUNNING);
org.apache.hadoop.mapreduce.v2.app.job.impl (3 usages found)
TestTaskAttempt (3 usages found)
verifyMillisCounters(Resource, int) (2 usages found)
361 app.waitForState(mTask, TaskState.RUNNING);
363 app.waitForState(rTask, TaskState.RUNNING);
testTaskAttemptAssignedKilledHistory(FailingAttemptsDuringAssignedMRApp) (1 usage found)
485 app.waitForState(task, TaskState.SCHEDULED);
org.apache.hadoop.mapreduce.v2.app.launcher (1 usage found)
TestContainerLauncher (1 usage found)
testSlowNM() (1 usage found)
296 app.waitForState(task, TaskState.SCHEDULED);
org.apache.hadoop.mapreduce.v2.app.rm (4 usages found)
TestRMContainerAllocator (4 usages found)
testReportedAppProgress() (2 usages found)
1072 mrApp.waitForState(t, TaskState.RUNNING);
1109 mrApp.waitForState(t, TaskState.RUNNING);
finishTask(DrainDispatcher, MockNM, MRApp, Task) (1 usage found)
1152 mrApp.waitForState(task, TaskState.SUCCEEDED);
testReportedAppProgressWithOnlyMaps() (1 usage found)
1221 mrApp.waitForState(t, TaskState.RUNNING);
--------------------------------------------------------------------------------------
Method
waitForState(Job, JobState)
Found usages (121 usages found)
org.apache.hadoop.mapreduce.v2 (2 usages found)
TestSpeculativeExecutionWithMRApp (2 usages found)
testSpeculateSuccessfulWithoutUpdateEvents() (1 usage found)
172 app.waitForState(job, JobState.RUNNING);
testSpeculateSuccessfulWithUpdateEvents() (1 usage found)
234 app.waitForState(job, JobState.RUNNING);
org.apache.hadoop.mapreduce.v2.app (89 usages found)
TestAMInfos (2 usages found)
testAMInfosWithoutRecoveryEnabled() (2 usages found)
49 app.waitForState(job, JobState.RUNNING);
73 app.waitForState(job, JobState.RUNNING);
TestFail (8 usages found)
testFailTask() (1 usage found)
71 app.waitForState(job, JobState.SUCCEEDED);
testMapFailureMaxPercent() (2 usages found)
101 app.waitForState(job, JobState.FAILED);
115 app.waitForState(job, JobState.SUCCEEDED);
testReduceFailureMaxPercent() (2 usages found)
132 app.waitForState(job, JobState.FAILED);
148 app.waitForState(job, JobState.SUCCEEDED);
testTimedOutTask() (1 usage found)
162 app.waitForState(job, JobState.FAILED);
testTaskFailWithUnusedContainer() (2 usages found)
188 app.waitForState(job, JobState.RUNNING);
203 app.waitForState(job, JobState.FAILED);
TestFetchFailure (7 usages found)
testFetchFailure() (2 usages found)
66 app.waitForState(job, JobState.RUNNING);
144 app.waitForState(job, JobState.SUCCEEDED);
testFetchFailureWithRecovery() (3 usages found)
198 app.waitForState(job, JobState.RUNNING);
251 app.waitForState(job, JobState.RUNNING);
278 app.waitForState(job, JobState.SUCCEEDED);
testFetchFailureMultipleReduces() (2 usages found)
291 app.waitForState(job, JobState.RUNNING);
396 app.waitForState(job, JobState.SUCCEEDED);
TestJobEndNotifier (3 usages found)
testAbsentNotificationOnNotLastRetryUnregistrationFailure() (2 usages found)
253 app.waitForState(job, JobState.RUNNING);
261 app.waitForState(job, JobState.RUNNING);
testNotificationOnLastRetryUnregistrationFailure() (1 usage found)
286 app.waitForState(job, JobState.RUNNING);
TestKill (7 usages found)
testKillJob() (1 usage found)
80 app.waitForState(job, JobState.KILLED);
testKillTask() (1 usage found)
124 app.waitForState(job, JobState.SUCCEEDED);
testKillTaskWait() (1 usage found)
196 app.waitForState(job, JobState.RUNNING);
testKillTaskWaitKillJobAfterTA_DONE() (1 usage found)
234 app.waitForState(job, JobState.RUNNING);
testKillTaskWaitKillJobBeforeTA_DONE() (1 usage found)
282 app.waitForState(job, JobState.RUNNING);
testKillTaskAttempt() (2 usages found)
371 app.waitForState(job, JobState.RUNNING);
393 app.waitForState(job, JobState.SUCCEEDED);
TestMRApp (19 usages found)
testMapReduce() (1 usage found)
84 app.waitForState(job, JobState.SUCCEEDED);
testZeroMaps() (1 usage found)
93 app.waitForState(job, JobState.SUCCEEDED);
testZeroMapReduces() (1 usage found)
101 app.waitForState(job, JobState.SUCCEEDED);
testCommitPending() (2 usages found)
108 app.waitForState(job, JobState.RUNNING);
140 app.waitForState(job, JobState.SUCCEEDED);
testCompletedMapsForReduceSlowstart() (2 usages found)
152 app.waitForState(job, JobState.RUNNING);
199 app.waitForState(job, JobState.SUCCEEDED);
testUpdatedNodes() (3 usages found)
226 app.waitForState(job1, JobState.RUNNING);
353 app.waitForState(job2, JobState.RUNNING);
440 app.waitForState(job2, JobState.SUCCEEDED);
testJobError() (2 usages found)
474 app.waitForState(job, JobState.RUNNING);
486 app.waitForState(job, JobState.ERROR);
testJobSuccess() (1 usage found)
499 app.waitForState(job, JobState.SUCCEEDED);
testJobRebootNotLastRetryOnUnregistrationFailure() (2 usages found)
507 app.waitForState(job, JobState.RUNNING);
519 app.waitForState(job, JobState.RUNNING);
testJobRebootOnLastRetryOnUnregistrationFailure() (2 usages found)
532 app.waitForState(job, JobState.RUNNING);
545 app.waitForState(job, JobState.RUNNING);
testCountersOnJobFinish() (1 usage found)
570 app.waitForState(job, JobState.SUCCEEDED);
testContainerPassThrough() (1 usage found)
617 app.waitForState(job, JobState.SUCCEEDED);
TestMRAppComponentDependencies (1 usage found)
testComponentStopOrder() (1 usage found)
47 app.waitForState(job, JobState.SUCCEEDED);
TestMRClientService (3 usages found)
test() (2 usages found)
84 app.waitForState(job, JobState.RUNNING);
186 app.waitForState(job, JobState.SUCCEEDED);
testViewAclOnlyCannotModify() (1 usage found)
209 app.waitForState(job, JobState.RUNNING);
TestRecovery (38 usages found)
testCrashed() (3 usages found)
146 app.waitForState(job, JobState.RUNNING);
261 app.waitForState(job, JobState.RUNNING);
297 app.waitForState(job, JobState.SUCCEEDED);
testCrashOfMapsOnlyJob() (3 usages found)
359 app.waitForState(job, JobState.RUNNING);
418 app.waitForState(job, JobState.RUNNING);
448 app.waitForState(job, JobState.SUCCEEDED);
testRecoverySuccessUsingCustomOutputCommitter() (3 usages found)
504 app.waitForState(job, JobState.RUNNING);
564 app.waitForState(job, JobState.RUNNING);
593 app.waitForState(job, JobState.SUCCEEDED);
testRecoveryWithSpillEncryption() (3 usages found)
613 app.waitForState(jobAttempt1, JobState.RUNNING);
639 app.waitForState(jobAttempt2, JobState.RUNNING);
660 app.waitForState(jobAttempt2, JobState.SUCCEEDED);
testRecoveryFailsUsingCustomOutputCommitter() (3 usages found)
683 app.waitForState(job, JobState.RUNNING);
743 app.waitForState(job, JobState.RUNNING);
786 app.waitForState(job, JobState.SUCCEEDED);
testMultipleCrashes() (4 usages found)
803 app.waitForState(job, JobState.RUNNING);
852 app.waitForState(job, JobState.RUNNING);
895 app.waitForState(job, JobState.RUNNING);
916 app.waitForState(job, JobState.SUCCEEDED);
testOutputRecovery() (3 usages found)
931 app.waitForState(job, JobState.RUNNING);
989 app.waitForState(job, JobState.RUNNING);
1024 app.waitForState(job, JobState.SUCCEEDED);
testPreviousJobOutputCleanedWhenNoRecovery() (2 usages found)
1042 app.waitForState(job, JobState.RUNNING);
1052 app.waitForState(job, JobState.RUNNING);
testPreviousJobIsNotCleanedWhenRecovery() (2 usages found)
1077 app.waitForState(job, JobState.RUNNING);
1087 app.waitForState(job, JobState.RUNNING);
testOutputRecoveryMapsOnly() (3 usages found)
1107 app.waitForState(job, JobState.RUNNING);
1155 app.waitForState(job, JobState.RUNNING);
1206 app.waitForState(job, JobState.SUCCEEDED);
testRecoveryWithOldCommiter() (3 usages found)
1222 app.waitForState(job, JobState.RUNNING);
1280 app.waitForState(job, JobState.RUNNING);
1315 app.waitForState(job, JobState.SUCCEEDED);
testSpeculative() (3 usages found)
1340 app.waitForState(job, JobState.RUNNING);
1415 app.waitForState(job, JobState.RUNNING);
1452 app.waitForState(job, JobState.SUCCEEDED);
testRecoveryWithoutShuffleSecret() (3 usages found)
1493 app.waitForState(job, JobState.RUNNING);
1540 app.waitForState(job, JobState.RUNNING);
1586 app.waitForState(job, JobState.SUCCEEDED);
TestStagingCleanup (1 usage found)
testStagingCleanupOrder() (1 usage found)
591 app.waitForState(job, JobState.SUCCEEDED);
org.apache.hadoop.mapreduce.v2.app.job.impl (12 usages found)
TestMapReduceChildJVM (7 usages found)
testCommandLine() (1 usage found)
56 app.waitForState(job, JobState.SUCCEEDED);
testReduceCommandLine(Configuration) (1 usage found)
112 app.waitForState(job, JobState.SUCCEEDED);
testCommandLineWithLog4JConifg() (1 usage found)
161 app.waitForState(job, JobState.SUCCEEDED);
testAutoHeapSize(int, int, String) (1 usage found)
230 app.waitForState(job, JobState.SUCCEEDED);
testEnvironmentVariables() (3 usages found)
288 app.waitForState(job, JobState.SUCCEEDED);
304 app.waitForState(job, JobState.SUCCEEDED);
318 app.waitForState(job, JobState.SUCCEEDED);
TestTaskAttempt (5 usages found)
verifyMillisCounters(Resource, int) (2 usages found)
356 app.waitForState(job, JobState.RUNNING);
380 app.waitForState(job, JobState.SUCCEEDED);
testMRAppHistory(MRApp) (1 usage found)
446 app.waitForState(job, JobState.FAILED);
testTaskAttemptAssignedFailHistory(FailingAttemptsDuringAssignedMRApp) (1 usage found)
472 app.waitForState(job, JobState.FAILED);
testTaskAttemptAssignedKilledHistory(FailingAttemptsDuringAssignedMRApp) (1 usage found)
482 app.waitForState(job, JobState.RUNNING);
org.apache.hadoop.mapreduce.v2.app.launcher (2 usages found)
TestContainerLauncher (2 usages found)
testSlowNM() (2 usages found)
290 app.waitForState(job, JobState.RUNNING);
307 app.waitForState(job, JobState.FAILED);
org.apache.hadoop.mapreduce.v2.app.webapp (4 usages found)
TestAMWebApp (4 usages found)
testMRWebAppSSLDisabled() (1 usage found)
229 app.waitForState(job, JobState.SUCCEEDED);
testMRWebAppSSLEnabled() (1 usage found)
286 app.waitForState(job, JobState.SUCCEEDED);
testMRWebAppSSLEnabledWithClientAuth() (1 usage found)
357 app.waitForState(job, JobState.SUCCEEDED);
testMRWebAppRedirection() (1 usage found)
411 app.waitForState(job, JobState.SUCCEEDED);
org.apache.hadoop.mapreduce.v2.hs (12 usages found)
TestJobHistoryEvents (3 usages found)
testHistoryEvents() (1 usage found)
62 app.waitForState(job, JobState.SUCCEEDED);
testEventsFlushOnStop() (1 usage found)
123 app.waitForState(job, JobState.SUCCEEDED);
testAssignedQueue() (1 usage found)
171 app.waitForState(job, JobState.SUCCEEDED);
TestJobHistoryParsing (8 usages found)
checkHistoryParsing(int, int, int) (1 usage found)
176 app.waitForState(job, JobState.SUCCEEDED);
testHistoryParsingForFailedAttempts() (1 usage found)
416 app.waitForState(job, JobState.SUCCEEDED);
testHistoryParsingForKilledAndFailedAttempts() (1 usage found)
492 app.waitForState(job, JobState.SUCCEEDED);
testCountersForFailedTask() (1 usage found)
551 app.waitForState(job, JobState.FAILED);
testDiagnosticsForKilledJob() (1 usage found)
614 app.waitForState(job, JobState.KILLED);
testScanningOldDirs() (1 usage found)
674 app.waitForState(job, JobState.SUCCEEDED);
testDeleteFileInfo() (1 usage found)
833 app.waitForState(job, JobState.SUCCEEDED);
testJobHistoryMethods() (1 usage found)
888 app.waitForState(job, JobState.SUCCEEDED);
TestJobHistoryServer (1 usage found)
testReports() (1 usage found)
106 app.waitForState(job, JobState.SUCCEEDED);
--------------------------------------------------------------------------------------
Method
waitForState(STATE)
Found usages (12 usages found)
org.apache.hadoop.mapreduce.v2.app (1 usage found)
TestKill (1 usage found)
testKillJob() (1 usage found)
83 app.waitForState(Service.STATE.STOPPED);
org.apache.hadoop.mapreduce.v2.hs (11 usages found)
TestJobHistoryEvents (3 usages found)
testHistoryEvents() (1 usage found)
65 app.waitForState(Service.STATE.STOPPED);
testEventsFlushOnStop() (1 usage found)
126 app.waitForState(Service.STATE.STOPPED);
testAssignedQueue() (1 usage found)
174 app.waitForState(Service.STATE.STOPPED);
TestJobHistoryParsing (8 usages found)
checkHistoryParsing(int, int, int) (1 usage found)
179 app.waitForState(Service.STATE.STOPPED);
testHistoryParsingForFailedAttempts() (1 usage found)
419 app.waitForState(Service.STATE.STOPPED);
testHistoryParsingForKilledAndFailedAttempts() (1 usage found)
495 app.waitForState(Service.STATE.STOPPED);
testCountersForFailedTask() (1 usage found)
554 app.waitForState(Service.STATE.STOPPED);
testDiagnosticsForKilledJob() (1 usage found)
617 app.waitForState(Service.STATE.STOPPED);
testScanningOldDirs() (1 usage found)
677 app.waitForState(Service.STATE.STOPPED);
testDeleteFileInfo() (1 usage found)
836 app.waitForState(Service.STATE.STOPPED);
testJobHistoryMethods() (1 usage found)
890 app.waitForState(Service.STATE.STOPPED);
--------------------------------------------------------------------------------------
Method
waitForState(STATE)
Found usages (12 usages found)
org.apache.hadoop.mapreduce.v2.app (1 usage found)
TestKill (1 usage found)
testKillJob() (1 usage found)
83 app.waitForState(Service.STATE.STOPPED);
org.apache.hadoop.mapreduce.v2.hs (11 usages found)
TestJobHistoryEvents (3 usages found)
testHistoryEvents() (1 usage found)
65 app.waitForState(Service.STATE.STOPPED);
testEventsFlushOnStop() (1 usage found)
126 app.waitForState(Service.STATE.STOPPED);
testAssignedQueue() (1 usage found)
174 app.waitForState(Service.STATE.STOPPED);
TestJobHistoryParsing (8 usages found)
checkHistoryParsing(int, int, int) (1 usage found)
179 app.waitForState(Service.STATE.STOPPED);
testHistoryParsingForFailedAttempts() (1 usage found)
419 app.waitForState(Service.STATE.STOPPED);
testHistoryParsingForKilledAndFailedAttempts() (1 usage found)
495 app.waitForState(Service.STATE.STOPPED);
testCountersForFailedTask() (1 usage found)
554 app.waitForState(Service.STATE.STOPPED);
testDiagnosticsForKilledJob() (1 usage found)
617 app.waitForState(Service.STATE.STOPPED);
testScanningOldDirs() (1 usage found)
677 app.waitForState(Service.STATE.STOPPED);
testDeleteFileInfo() (1 usage found)
836 app.waitForState(Service.STATE.STOPPED);
testJobHistoryMethods() (1 usage found)
890 app.waitForState(Service.STATE.STOPPED);
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org