You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Eli Collins <el...@cloudera.com> on 2011/12/15 06:51:09 UTC
test failures on trunk
Hey gang,
Looks like a number of the trunk tests
(https://builds.apache.org/job/Hadoop-Hdfs-trunk) started failing on
the 10th, due to the following. Ring a bell? Maybe due to all the
recent protocol changes?
Error Message
org.apache.hadoop.hdfs.protocol.HdfsFileStatus cannot be cast to
org.apache.hadoop.hdfs.protocol.HdfsLocatedFileStatus
Stacktrace
java.lang.ClassCastException:
org.apache.hadoop.hdfs.protocol.HdfsFileStatus cannot be cast to
org.apache.hadoop.hdfs.protocol.HdfsLocatedFileStatus
at org.apache.hadoop.hdfs.DistributedFileSystem$1.hasNext(DistributedFileSystem.java:452)
at org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:1551)
at org.apache.hadoop.fs.FileSystem$5.next(FileSystem.java:1581)
at org.apache.hadoop.fs.FileSystem$5.next(FileSystem.java:1541)
at org.apache.hadoop.fs.TestListFiles.testDirectory(TestListFiles.java:146)
Thanks,
Eli
RE: test failures on trunk
Posted by Uma Maheswara Rao G <ma...@huawei.com>.
Thanks a lot, Suresh.
Here is the Jira for this issue https://issues.apache.org/jira/browse/HDFS-2687
Regards,
Uma
________________________________________
From: Suresh Srinivas [suresh@hortonworks.com]
Sent: Thursday, December 15, 2011 9:27 PM
To: hdfs-dev@hadoop.apache.org
Subject: Re: test failures on trunk
I will fix this bug today.
On Thursday, December 15, 2011, Uma Maheswara Rao G <ma...@huawei.com>
wrote:
> Yes Eli, Agree with you.
>
> I think we must take a look immediately now.
>
> I just debugged the failure related to this class cast exception.
>
> Following is the initial analysis:
>
> Previous behaviour of getListing in ClientNamenodeProtocolTranslatorR23 is
> It will pass needLocation flag as true and server side will be packed
empty LocatedBlocks with HdfsLocatedFileStatus object if getBlocations are
null and needLocation flag is true. So, cleint will reconstruct the
HdfsLocatedFileStatus directly if HdfsfileStatus is instanceOf
HdfsLocatedFileStatus.
>
> But with new implementation of protocolBuffers, It is just ignoring the
LocatedBlocks are server side if the getBlocklocations are null. At client
side, to reconstruct the HDFSFileStatus object, it is depending on
locations field. Since there is no locations field here, it is considering
HDFSFileStatus as just HDFSFileStatus.
>
> Here test assumes that HDFSFileStatus obj will be type of
HdfsLocatedFileStatus and trying cast. This is failing here.
>
> I think Suresh and Sanjay can give more hint on this.
>
> Regards,
> Uma
> ________________________________________
> From: Eli Collins [eli@cloudera.com]
> Sent: Thursday, December 15, 2011 11:21 AM
> To: hdfs-dev@hadoop.apache.org
> Subject: test failures on trunk
>
> Hey gang,
>
> Looks like a number of the trunk tests
> (https://builds.apache.org/job/Hadoop-Hdfs-trunk) started failing on
> the 10th, due to the following. Ring a bell? Maybe due to all the
> recent protocol changes?
>
> Error Message
>
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus cannot be cast to
> org.apache.hadoop.hdfs.protocol.HdfsLocatedFileStatus
> Stacktrace
>
> java.lang.ClassCastException:
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus cannot be cast to
> org.apache.hadoop.hdfs.protocol.HdfsLocatedFileStatus
> at
org.apache.hadoop.hdfs.DistributedFileSystem$1.hasNext(DistributedFileSystem.java:452)
> at org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:1551)
> at org.apache.hadoop.fs.FileSystem$5.next(FileSystem.java:1581)
> at org.apache.hadoop.fs.FileSystem$5.next(FileSystem.java:1541)
> at
org.apache.hadoop.fs.TestListFiles.testDirectory(TestListFiles.java:146)
>
> Thanks,
> Eli
>
Re: test failures on trunk
Posted by Suresh Srinivas <su...@hortonworks.com>.
I will fix this bug today.
On Thursday, December 15, 2011, Uma Maheswara Rao G <ma...@huawei.com>
wrote:
> Yes Eli, Agree with you.
>
> I think we must take a look immediately now.
>
> I just debugged the failure related to this class cast exception.
>
> Following is the initial analysis:
>
> Previous behaviour of getListing in ClientNamenodeProtocolTranslatorR23 is
> It will pass needLocation flag as true and server side will be packed
empty LocatedBlocks with HdfsLocatedFileStatus object if getBlocations are
null and needLocation flag is true. So, cleint will reconstruct the
HdfsLocatedFileStatus directly if HdfsfileStatus is instanceOf
HdfsLocatedFileStatus.
>
> But with new implementation of protocolBuffers, It is just ignoring the
LocatedBlocks are server side if the getBlocklocations are null. At client
side, to reconstruct the HDFSFileStatus object, it is depending on
locations field. Since there is no locations field here, it is considering
HDFSFileStatus as just HDFSFileStatus.
>
> Here test assumes that HDFSFileStatus obj will be type of
HdfsLocatedFileStatus and trying cast. This is failing here.
>
> I think Suresh and Sanjay can give more hint on this.
>
> Regards,
> Uma
> ________________________________________
> From: Eli Collins [eli@cloudera.com]
> Sent: Thursday, December 15, 2011 11:21 AM
> To: hdfs-dev@hadoop.apache.org
> Subject: test failures on trunk
>
> Hey gang,
>
> Looks like a number of the trunk tests
> (https://builds.apache.org/job/Hadoop-Hdfs-trunk) started failing on
> the 10th, due to the following. Ring a bell? Maybe due to all the
> recent protocol changes?
>
> Error Message
>
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus cannot be cast to
> org.apache.hadoop.hdfs.protocol.HdfsLocatedFileStatus
> Stacktrace
>
> java.lang.ClassCastException:
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus cannot be cast to
> org.apache.hadoop.hdfs.protocol.HdfsLocatedFileStatus
> at
org.apache.hadoop.hdfs.DistributedFileSystem$1.hasNext(DistributedFileSystem.java:452)
> at org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:1551)
> at org.apache.hadoop.fs.FileSystem$5.next(FileSystem.java:1581)
> at org.apache.hadoop.fs.FileSystem$5.next(FileSystem.java:1541)
> at
org.apache.hadoop.fs.TestListFiles.testDirectory(TestListFiles.java:146)
>
> Thanks,
> Eli
>
RE: test failures on trunk
Posted by Uma Maheswara Rao G <ma...@huawei.com>.
Yes Eli, Agree with you.
I think we must take a look immediately now.
I just debugged the failure related to this class cast exception.
Following is the initial analysis:
Previous behaviour of getListing in ClientNamenodeProtocolTranslatorR23 is
It will pass needLocation flag as true and server side will be packed empty LocatedBlocks with HdfsLocatedFileStatus object if getBlocations are null and needLocation flag is true. So, cleint will reconstruct the HdfsLocatedFileStatus directly if HdfsfileStatus is instanceOf HdfsLocatedFileStatus.
But with new implementation of protocolBuffers, It is just ignoring the LocatedBlocks are server side if the getBlocklocations are null. At client side, to reconstruct the HDFSFileStatus object, it is depending on locations field. Since there is no locations field here, it is considering HDFSFileStatus as just HDFSFileStatus.
Here test assumes that HDFSFileStatus obj will be type of HdfsLocatedFileStatus and trying cast. This is failing here.
I think Suresh and Sanjay can give more hint on this.
Regards,
Uma
________________________________________
From: Eli Collins [eli@cloudera.com]
Sent: Thursday, December 15, 2011 11:21 AM
To: hdfs-dev@hadoop.apache.org
Subject: test failures on trunk
Hey gang,
Looks like a number of the trunk tests
(https://builds.apache.org/job/Hadoop-Hdfs-trunk) started failing on
the 10th, due to the following. Ring a bell? Maybe due to all the
recent protocol changes?
Error Message
org.apache.hadoop.hdfs.protocol.HdfsFileStatus cannot be cast to
org.apache.hadoop.hdfs.protocol.HdfsLocatedFileStatus
Stacktrace
java.lang.ClassCastException:
org.apache.hadoop.hdfs.protocol.HdfsFileStatus cannot be cast to
org.apache.hadoop.hdfs.protocol.HdfsLocatedFileStatus
at org.apache.hadoop.hdfs.DistributedFileSystem$1.hasNext(DistributedFileSystem.java:452)
at org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:1551)
at org.apache.hadoop.fs.FileSystem$5.next(FileSystem.java:1581)
at org.apache.hadoop.fs.FileSystem$5.next(FileSystem.java:1541)
at org.apache.hadoop.fs.TestListFiles.testDirectory(TestListFiles.java:146)
Thanks,
Eli