You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by "Camp, Roy" <rc...@ebay.com> on 2013/01/15 01:10:17 UTC
flume-ng data recovery
I somehow got one of my instances into a bad state where I continue to get the following error. I have two data log files with about 2GB worth of data that I'd like to save. Removing the current checkpoint and/or inflight files did not work. Any ideas?
Thanks,
Roy
Flume-NG 1.3 SNAPSHOT - not the 1.3 stable release
2013-01-14 17:03:53,570 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:355)] Replaying logs with v2 replay logic
2013-01-14 17:03:53,574 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:223)] Starting replay of [/var/log/flume-ng/collectorfix/data/log-240, /var/log/flume-ng/collectorfix/data/log-241]
2013-01-14 17:03:53,576 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.channel.file.Log.replay(Log.java:373)] Failed to initialize Log on [channel=collectorfilefix]
java.lang.IllegalStateException: Checksum of inflights file differs from the checksum expected.
at org.apache.flume.channel.file.FlumeEventQueue$InflightEventWrapper.deserialize(FlumeEventQueue.java:487)
at org.apache.flume.channel.file.FlumeEventQueue.deserializeInflightPuts(FlumeEventQueue.java:83)
at org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:226)
at org.apache.flume.channel.file.Log.replay(Log.java:356)
at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2013-01-14 17:03:53,579 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.channel.file.FileChannel.start(FileChannel.java:269)] Failed to start the file channel [channel=collectorfilefix]
java.lang.IllegalStateException: Checksum of inflights file differs from the checksum expected.
at org.apache.flume.channel.file.FlumeEventQueue$InflightEventWrapper.deserialize(FlumeEventQueue.java:487)
at org.apache.flume.channel.file.FlumeEventQueue.deserializeInflightPuts(FlumeEventQueue.java:83)
at org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:226)
at org.apache.flume.channel.file.Log.replay(Log.java:356)
at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Re: flume-ng data recovery
Posted by Brock Noland <br...@cloudera.com>.
Hi,
OK..... I would increase the capacity of the channel to say 2000000
with the original unmodified files.
I would also upgrade to the latest 1.3.1 since there are many file
channel fixes in 1.3.0 and 1.3.1.
On Mon, Jan 14, 2013 at 5:33 PM, Camp, Roy <rc...@ebay.com> wrote:
> When I deleted both, the error changes to the one below. However, I removed data file log-241 and was able to replay log-240 with no problem. After that completed I removed the log-240 and put the log-241 in the data directory. It didn't appear to be working but I split log-241 into chunks and the first chunks seem to be working so far.
>
> Thanks,
>
> Roy
>
>
>
> 2013-01-14 17:29:00,346 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:304)] Found NextFileID 241, from [/var/log/flume-ng/collectorfix/data/log-240, /var/log/flume-ng/collectorfix/data/log-241]
> 2013-01-14 17:29:00,381 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFile.<init>(EventQueueBackingStoreFile.java:71)] Preallocated /var/log/flume-ng/collectorfix/checkpoint/checkpoint to 8008232 for capacity 1000000
> 2013-01-14 17:29:00,384 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFileV3.<init>(EventQueueBackingStoreFileV3.java:46)] Starting up with /var/log/flume-ng/collectorfix/checkpoint/checkpoint and /var/log/flume-ng/collectorfix/checkpoint/checkpoint.meta
> 2013-01-14 17:29:00,442 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:336)] Last Checkpoint Mon Jan 14 17:29:00 GMT-07:00 2013, queue depth = 0
> 2013-01-14 17:29:00,454 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:355)] Replaying logs with v2 replay logic
> 2013-01-14 17:29:00,458 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:223)] Starting replay of [/var/log/flume-ng/collectorfix/data/log-240, /var/log/flume-ng/collectorfix/data/log-241]
> 2013-01-14 17:29:00,459 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:236)] Replaying /var/log/flume-ng/collectorfix/data/log-240
> 2013-01-14 17:29:00,474 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.tools.DirectMemoryUtils.getDefaultDirectMemorySize(DirectMemoryUtils.java:113)] Unable to get maxDirectMemory from VM: NoSuchMethodException: sun.misc.VM.maxDirectMemory(null)
> 2013-01-14 17:29:00,477 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.tools.DirectMemoryUtils.allocate(DirectMemoryUtils.java:47)] Direct Memory Allocation: Allocation = 1048576, Allocated = 0, MaxDirectMemorySize = 2033909760, Remaining = 2033909760
> 2013-01-14 17:29:00,527 (lifecycleSupervisor-1-1) [WARN - org.apache.flume.channel.file.LogFile$SequentialReader.skipToLastCheckpointPosition(LogFile.java:431)] Checkpoint for file(/var/log/flume-ng/collectorfix/data/log-240) is: 1355687437770, which is beyond the requested checkpoint time: 0 and position 1621631818
> 2013-01-14 17:29:00,548 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:236)] Replaying /var/log/flume-ng/collectorfix/data/log-241
> 2013-01-14 17:29:00,548 (lifecycleSupervisor-1-1) [WARN - org.apache.flume.channel.file.LogFile$SequentialReader.skipToLastCheckpointPosition(LogFile.java:431)] Checkpoint for file(/var/log/flume-ng/collectorfix/data/log-241) is: 1355687437770, which is beyond the requested checkpoint time: 0 and position 490073589
> 2013-01-14 17:29:40,621 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:452)] Encountered EOF at 1623187930 in /var/log/flume-ng/collectorfix/data/log-240
> 2013-01-14 17:29:48,183 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.channel.file.Log.replay(Log.java:373)] Failed to initialize Log on [channel=collectorfilefix]
> java.lang.IllegalStateException: Unable to add FlumeEventPointer [fileID=241, offset=510579142]. Queue depth = 1000000, Capacity = 1000000
> at org.apache.flume.channel.file.ReplayHandler.processCommit(ReplayHandler.java:374)
> at org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:309)
> at org.apache.flume.channel.file.Log.replay(Log.java:356)
> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
> at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
> at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> 2013-01-14 17:29:48,184 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.channel.file.FileChannel.start(FileChannel.java:269)] Failed to start the file channel [channel=collectorfilefix]
> java.lang.IllegalStateException: Unable to add FlumeEventPointer [fileID=241, offset=510579142]. Queue depth = 1000000, Capacity = 1000000
> at org.apache.flume.channel.file.ReplayHandler.processCommit(ReplayHandler.java:374)
> at org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:309)
> at org.apache.flume.channel.file.Log.replay(Log.java:356)
> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
> at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
> at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
>
>
>
>
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
RE: flume-ng data recovery
Posted by "Camp, Roy" <rc...@ebay.com>.
When I deleted both, the error changes to the one below. However, I removed data file log-241 and was able to replay log-240 with no problem. After that completed I removed the log-240 and put the log-241 in the data directory. It didn't appear to be working but I split log-241 into chunks and the first chunks seem to be working so far.
Thanks,
Roy
2013-01-14 17:29:00,346 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:304)] Found NextFileID 241, from [/var/log/flume-ng/collectorfix/data/log-240, /var/log/flume-ng/collectorfix/data/log-241]
2013-01-14 17:29:00,381 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFile.<init>(EventQueueBackingStoreFile.java:71)] Preallocated /var/log/flume-ng/collectorfix/checkpoint/checkpoint to 8008232 for capacity 1000000
2013-01-14 17:29:00,384 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFileV3.<init>(EventQueueBackingStoreFileV3.java:46)] Starting up with /var/log/flume-ng/collectorfix/checkpoint/checkpoint and /var/log/flume-ng/collectorfix/checkpoint/checkpoint.meta
2013-01-14 17:29:00,442 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:336)] Last Checkpoint Mon Jan 14 17:29:00 GMT-07:00 2013, queue depth = 0
2013-01-14 17:29:00,454 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:355)] Replaying logs with v2 replay logic
2013-01-14 17:29:00,458 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:223)] Starting replay of [/var/log/flume-ng/collectorfix/data/log-240, /var/log/flume-ng/collectorfix/data/log-241]
2013-01-14 17:29:00,459 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:236)] Replaying /var/log/flume-ng/collectorfix/data/log-240
2013-01-14 17:29:00,474 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.tools.DirectMemoryUtils.getDefaultDirectMemorySize(DirectMemoryUtils.java:113)] Unable to get maxDirectMemory from VM: NoSuchMethodException: sun.misc.VM.maxDirectMemory(null)
2013-01-14 17:29:00,477 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.tools.DirectMemoryUtils.allocate(DirectMemoryUtils.java:47)] Direct Memory Allocation: Allocation = 1048576, Allocated = 0, MaxDirectMemorySize = 2033909760, Remaining = 2033909760
2013-01-14 17:29:00,527 (lifecycleSupervisor-1-1) [WARN - org.apache.flume.channel.file.LogFile$SequentialReader.skipToLastCheckpointPosition(LogFile.java:431)] Checkpoint for file(/var/log/flume-ng/collectorfix/data/log-240) is: 1355687437770, which is beyond the requested checkpoint time: 0 and position 1621631818
2013-01-14 17:29:00,548 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:236)] Replaying /var/log/flume-ng/collectorfix/data/log-241
2013-01-14 17:29:00,548 (lifecycleSupervisor-1-1) [WARN - org.apache.flume.channel.file.LogFile$SequentialReader.skipToLastCheckpointPosition(LogFile.java:431)] Checkpoint for file(/var/log/flume-ng/collectorfix/data/log-241) is: 1355687437770, which is beyond the requested checkpoint time: 0 and position 490073589
2013-01-14 17:29:40,621 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:452)] Encountered EOF at 1623187930 in /var/log/flume-ng/collectorfix/data/log-240
2013-01-14 17:29:48,183 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.channel.file.Log.replay(Log.java:373)] Failed to initialize Log on [channel=collectorfilefix]
java.lang.IllegalStateException: Unable to add FlumeEventPointer [fileID=241, offset=510579142]. Queue depth = 1000000, Capacity = 1000000
at org.apache.flume.channel.file.ReplayHandler.processCommit(ReplayHandler.java:374)
at org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:309)
at org.apache.flume.channel.file.Log.replay(Log.java:356)
at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2013-01-14 17:29:48,184 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.channel.file.FileChannel.start(FileChannel.java:269)] Failed to start the file channel [channel=collectorfilefix]
java.lang.IllegalStateException: Unable to add FlumeEventPointer [fileID=241, offset=510579142]. Queue depth = 1000000, Capacity = 1000000
at org.apache.flume.channel.file.ReplayHandler.processCommit(ReplayHandler.java:374)
at org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:309)
at org.apache.flume.channel.file.Log.replay(Log.java:356)
at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Re: flume-ng data recovery
Posted by Brock Noland <br...@cloudera.com>.
Roy,
You deleted the checkpoint files AND the inflights and you got the same error?
Brock
On Mon, Jan 14, 2013 at 4:10 PM, Camp, Roy <rc...@ebay.com> wrote:
> I somehow got one of my instances into a bad state where I continue to get
> the following error. I have two data log files with about 2GB worth of data
> that I’d like to save. Removing the current checkpoint and/or inflight
> files did not work. Any ideas?
>
>
>
> Thanks,
>
>
>
> Roy
>
>
>
>
>
> Flume-NG 1.3 SNAPSHOT – not the 1.3 stable release
>
>
>
> 2013-01-14 17:03:53,570 (lifecycleSupervisor-1-1) [INFO -
> org.apache.flume.channel.file.Log.replay(Log.java:355)] Replaying logs with
> v2 replay logic
>
> 2013-01-14 17:03:53,574 (lifecycleSupervisor-1-1) [INFO -
> org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:223)]
> Starting replay of [/var/log/flume-ng/collectorfix/data/log-240,
> /var/log/flume-ng/collectorfix/data/log-241]
>
> 2013-01-14 17:03:53,576 (lifecycleSupervisor-1-1) [ERROR -
> org.apache.flume.channel.file.Log.replay(Log.java:373)] Failed to initialize
> Log on [channel=collectorfilefix]
>
> java.lang.IllegalStateException: Checksum of inflights file differs from the
> checksum expected.
>
> at
> org.apache.flume.channel.file.FlumeEventQueue$InflightEventWrapper.deserialize(FlumeEventQueue.java:487)
>
> at
> org.apache.flume.channel.file.FlumeEventQueue.deserializeInflightPuts(FlumeEventQueue.java:83)
>
> at
> org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:226)
>
> at org.apache.flume.channel.file.Log.replay(Log.java:356)
>
> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
>
> at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
>
> at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
> Source)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
> Source)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>
> at java.lang.Thread.run(Unknown Source)
>
> 2013-01-14 17:03:53,579 (lifecycleSupervisor-1-1) [ERROR -
> org.apache.flume.channel.file.FileChannel.start(FileChannel.java:269)]
> Failed to start the file channel [channel=collectorfilefix]
>
> java.lang.IllegalStateException: Checksum of inflights file differs from the
> checksum expected.
>
> at
> org.apache.flume.channel.file.FlumeEventQueue$InflightEventWrapper.deserialize(FlumeEventQueue.java:487)
>
> at
> org.apache.flume.channel.file.FlumeEventQueue.deserializeInflightPuts(FlumeEventQueue.java:83)
>
> at
> org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:226)
>
> at org.apache.flume.channel.file.Log.replay(Log.java:356)
>
> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
>
> at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
>
> at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
> Source)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
> Source)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>
> at java.lang.Thread.run(Unknown Source)
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/