You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by Renato Marroquín Mogrovejo <re...@gmail.com> on 2013/10/02 15:17:19 UTC
Re: File access error stack about DFS directory from running unit tests
Hi Henry,
I don´t know where this is, but maybe if you could point me to the test
breaking, I will give the code a look. I have been playing around with the
Gora-MapReduce and man, we need to sit down and do some coding ;)
Renato M.
2013/9/28 Henry Saputra <he...@gmail.com>
> Hi Guys,
>
> I am seeing some file access error in the console log when running the
> unit tests, I assume this is coming from MapReduce test case:
>
> Starting DataNode 0 with dfs.data.dir:
>
> /home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data1,/home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data2
> 13/09/27 23:28:03 WARN impl.MetricsSystemImpl: Metrics system not
> started: Cannot locate configuration: tried
> hadoop-metrics2-datanode.properties, hadoop-metrics2.properties
> 13/09/27 23:28:04 WARN datanode.DataNode: Invalid directory in
> dfs.data.dir: Incorrect permission for
> /home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data1,
> expected: rwxr-xr-x, while actual: rwxrwxr-x
> 13/09/27 23:28:04 WARN datanode.DataNode: Invalid directory in
> dfs.data.dir: Incorrect permission for
> /home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data2,
> expected: rwxr-xr-x, while actual: rwxrwxr-x
> 13/09/27 23:28:04 ERROR datanode.DataNode: All directories in
> dfs.data.dir are invalid.
> 12910 [main] ERROR
> org.apache.gora.mapreduce.DataStoreMapReduceTestBase - Hadoop Test
> Case set up failed
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
> at
> org.apache.hadoop.mapred.HadoopTestCase.setUp(HadoopTestCase.java:148)
> at
> org.apache.gora.mapreduce.DataStoreMapReduceTestBase.setUp(DataStoreMapReduceTestBase.java:61)
> at junit.framework.TestCase.runBare(TestCase.java:132)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at
> junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at
> junit.framework.TestSuite.runTest(TestSuite.java:243)
> at
> junit.framework.TestSuite.run(TestSuite.java:238)
>
>
> Anyone know where do we set conf for the mapreduce test case and where
> does Gora create the dfs.data.dir directory for the DFS to setup?
>
> - Henry
>
Re: File access error stack about DFS directory from running unit tests
Posted by Henry Saputra <he...@gmail.com>.
Hey, thanks Renato.
The funny thing is that the whole tests passed in the final report. I
found this stack by saving the console output and start finding for
"failed" and "ERROR".
- Henry
On Wed, Oct 2, 2013 at 6:18 AM, Renato Marroquín Mogrovejo
<re...@gmail.com> wrote:
> Sorry, my bad, I overlooked the stack-trace. I will get back to you
> hopefully by the end of the day.
>
>
> Renato M.
>
>
> 2013/10/2 Renato Marroquín Mogrovejo <re...@gmail.com>
>
>> Hi Henry,
>>
>> I don´t know where this is, but maybe if you could point me to the test
>> breaking, I will give the code a look. I have been playing around with the
>> Gora-MapReduce and man, we need to sit down and do some coding ;)
>>
>>
>> Renato M.
>>
>>
>> 2013/9/28 Henry Saputra <he...@gmail.com>
>>
>>> Hi Guys,
>>>
>>> I am seeing some file access error in the console log when running the
>>> unit tests, I assume this is coming from MapReduce test case:
>>>
>>> Starting DataNode 0 with dfs.data.dir:
>>>
>>> /home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data1,/home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data2
>>> 13/09/27 23:28:03 WARN impl.MetricsSystemImpl: Metrics system not
>>> started: Cannot locate configuration: tried
>>> hadoop-metrics2-datanode.properties, hadoop-metrics2.properties
>>> 13/09/27 23:28:04 WARN datanode.DataNode: Invalid directory in
>>> dfs.data.dir: Incorrect permission for
>>> /home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data1,
>>> expected: rwxr-xr-x, while actual: rwxrwxr-x
>>> 13/09/27 23:28:04 WARN datanode.DataNode: Invalid directory in
>>> dfs.data.dir: Incorrect permission for
>>> /home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data2,
>>> expected: rwxr-xr-x, while actual: rwxrwxr-x
>>> 13/09/27 23:28:04 ERROR datanode.DataNode: All directories in
>>> dfs.data.dir are invalid.
>>> 12910 [main] ERROR
>>> org.apache.gora.mapreduce.DataStoreMapReduceTestBase - Hadoop Test
>>> Case set up failed
>>> java.lang.NullPointerException
>>> at
>>> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
>>> at
>>> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
>>> at
>>> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
>>> at
>>> org.apache.hadoop.mapred.HadoopTestCase.setUp(HadoopTestCase.java:148)
>>> at
>>> org.apache.gora.mapreduce.DataStoreMapReduceTestBase.setUp(DataStoreMapReduceTestBase.java:61)
>>> at junit.framework.TestCase.runBare(TestCase.java:132)
>>> at junit.framework.TestResult$1.protect(TestResult.java:110)
>>> at
>>> junit.framework.TestResult.runProtected(TestResult.java:128)
>>> at junit.framework.TestResult.run(TestResult.java:113)
>>> at junit.framework.TestCase.run(TestCase.java:124)
>>> at
>>> junit.framework.TestSuite.runTest(TestSuite.java:243)
>>> at
>>> junit.framework.TestSuite.run(TestSuite.java:238)
>>>
>>>
>>> Anyone know where do we set conf for the mapreduce test case and where
>>> does Gora create the dfs.data.dir directory for the DFS to setup?
>>>
>>> - Henry
>>>
>>
>>
Re: File access error stack about DFS directory from running unit tests
Posted by Renato Marroquín Mogrovejo <re...@gmail.com>.
Sorry, my bad, I overlooked the stack-trace. I will get back to you
hopefully by the end of the day.
Renato M.
2013/10/2 Renato Marroquín Mogrovejo <re...@gmail.com>
> Hi Henry,
>
> I don´t know where this is, but maybe if you could point me to the test
> breaking, I will give the code a look. I have been playing around with the
> Gora-MapReduce and man, we need to sit down and do some coding ;)
>
>
> Renato M.
>
>
> 2013/9/28 Henry Saputra <he...@gmail.com>
>
>> Hi Guys,
>>
>> I am seeing some file access error in the console log when running the
>> unit tests, I assume this is coming from MapReduce test case:
>>
>> Starting DataNode 0 with dfs.data.dir:
>>
>> /home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data1,/home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data2
>> 13/09/27 23:28:03 WARN impl.MetricsSystemImpl: Metrics system not
>> started: Cannot locate configuration: tried
>> hadoop-metrics2-datanode.properties, hadoop-metrics2.properties
>> 13/09/27 23:28:04 WARN datanode.DataNode: Invalid directory in
>> dfs.data.dir: Incorrect permission for
>> /home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data1,
>> expected: rwxr-xr-x, while actual: rwxrwxr-x
>> 13/09/27 23:28:04 WARN datanode.DataNode: Invalid directory in
>> dfs.data.dir: Incorrect permission for
>> /home/hsaputra/open/gora/trunk/gora-core/target/test-data/dfs/data/data2,
>> expected: rwxr-xr-x, while actual: rwxrwxr-x
>> 13/09/27 23:28:04 ERROR datanode.DataNode: All directories in
>> dfs.data.dir are invalid.
>> 12910 [main] ERROR
>> org.apache.gora.mapreduce.DataStoreMapReduceTestBase - Hadoop Test
>> Case set up failed
>> java.lang.NullPointerException
>> at
>> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
>> at
>> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
>> at
>> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
>> at
>> org.apache.hadoop.mapred.HadoopTestCase.setUp(HadoopTestCase.java:148)
>> at
>> org.apache.gora.mapreduce.DataStoreMapReduceTestBase.setUp(DataStoreMapReduceTestBase.java:61)
>> at junit.framework.TestCase.runBare(TestCase.java:132)
>> at junit.framework.TestResult$1.protect(TestResult.java:110)
>> at
>> junit.framework.TestResult.runProtected(TestResult.java:128)
>> at junit.framework.TestResult.run(TestResult.java:113)
>> at junit.framework.TestCase.run(TestCase.java:124)
>> at
>> junit.framework.TestSuite.runTest(TestSuite.java:243)
>> at
>> junit.framework.TestSuite.run(TestSuite.java:238)
>>
>>
>> Anyone know where do we set conf for the mapreduce test case and where
>> does Gora create the dfs.data.dir directory for the DFS to setup?
>>
>> - Henry
>>
>
>