1.4 unit test failing in Bamboo

[Mike Keane <mk...@dotomi.com>]

I've done a small proof of concept on some flume sink customizations in 1.3.1.  I want to incorporate these customizations into our build server (Bamboo).  I started by creating a local git repo of 1.4 and I'm able to run a mvn clean install just fine.  However when I push the raw un-altered code to our build server the build fails and not consistently.

Half the time TestExecSource fails in flume-ng-core:

build    01-Aug-2013 22:56:11    testMonitoredCounterGroup(org.apache.flume.source.TestExecSource)  Time elapsed: 1264 sec  <<< FAILURE!
build    01-Aug-2013 22:56:11    org.junit.ComparisonFailure: Expected Value: 5 expected:<[5]> but was:<[0]>
build    01-Aug-2013 22:56:11        at org.junit.Assert.assertEquals(Assert.java:125)
build    01-Aug-2013 22:56:11        at org.apache.flume.source.TestExecSource.testMonitoredCounterGroup(TestExecSource.java:215)

When that passes my build fails in flume-hdfs-sink


build    02-Aug-2013 00:03:30    simpleHDFSGZipCompressedTest(org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster)  Time elapsed: 2676 sec  <<< ERROR!
build    02-Aug-2013 00:03:30    java.lang.NullPointerException
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
build    02-Aug-2013 00:03:30        at org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster.simpleHDFSGZipCompressedTest(TestHDFSEventSinkOnMiniCluster.java:164)
...
build    02-Aug-2013 00:03:30    underReplicationTest(org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster)  Time elapsed: 31 sec  <<< ERROR!
build    02-Aug-2013 00:03:30    java.io.IOException: Cannot lock storage target/test/dfs/dfs/name1. The directory is already locked.
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:602)
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1219)
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1237)
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1164)
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:184)
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:267)
build    02-Aug-2013 00:03:30        at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)


I'm at a loss to explain the inconsistency of flume-ng-core and when it does succeed how to solve the failure of HDFS-sink.  Any advice would be appreciated.


Thanks,

Mike




This email and any files included with it may contain privileged,
proprietary and/or confidential information that is for the sole use
of the intended recipient(s).  Any disclosure, copying, distribution,
posting, or use of the information contained in or attached to this
email is prohibited unless permitted by the sender.  If you have
received this email in error, please immediately notify the sender
via return email, telephone, or fax and destroy this original transmission
and its included files without reading or saving it in any manner.
Thank you.