You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Allan Yan (JIRA)" <ji...@apache.org> on 2013/07/17 08:44:49 UTC
[jira] [Commented] (HADOOP-9740) FsShell's Text command does not
read avro data files stored on HDFS
[ https://issues.apache.org/jira/browse/HADOOP-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13710796#comment-13710796 ]
Allan Yan commented on HADOOP-9740:
-----------------------------------
My environment is Mac OS X 10.7.5, jdk1.6.0_45.
Trunk build failed and complain class LogCapturer not found. However, the test files were build succeeded. Not sure why it could not find the LogCapturer class.
{code}
hyan-mbp:trunk hyan$ grep -R 'class LogCapturer' .
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/.svn/text-base/GenericTestUtils.java.svn-base: public static class LogCapturer {
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java: public static class LogCapturer {
hyan-mbp:trunk hyan$ ls hadoop-common-project/hadoop-common/target/
antrun generated-sources hadoop-common-0.24.0-SNAPSHOT-tests.jar hadoop-common-3.0.0-SNAPSHOT-tests.jar surefire-reports
classes generated-test-sources hadoop-common-0.24.0-SNAPSHOT.jar hadoop-common-3.0.0-SNAPSHOT.jar test
compile-proto.sh hadoop-common-0.24.0-SNAPSHOT-sources.jar hadoop-common-3.0.0-SNAPSHOT-sources.jar log test-classes
compile-test-proto.sh hadoop-common-0.24.0-SNAPSHOT-test-sources.jar hadoop-common-3.0.0-SNAPSHOT-test-sources.jar maven-archiver test-dir
hyan-mbp:trunk hyan$ ls ~/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-SNAPSHOT/
_maven.repositories hadoop-common-3.0.0-SNAPSHOT-test-sources.jar hadoop-common-3.0.0-SNAPSHOT.jar maven-metadata-local.xml
hadoop-common-3.0.0-SNAPSHOT-sources.jar hadoop-common-3.0.0-SNAPSHOT-tests.jar hadoop-common-3.0.0-SNAPSHOT.pom
{code}
> FsShell's Text command does not read avro data files stored on HDFS
> -------------------------------------------------------------------
>
> Key: HADOOP-9740
> URL: https://issues.apache.org/jira/browse/HADOOP-9740
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.0.5-alpha
> Reporter: Allan Yan
> Labels: patch
> Attachments: HADOOP-9740.patch, maven_unit_test_error.log
>
>
> HADOOP-8597 added support for reading avro data files from FsShell Text command. However, it does not work with files stored on HDFS. Here is the error message:
> {code}
> $hadoop fs -text hdfs://localhost:8020/test.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> {code}
> The problem is because the File constructor complains not able to recognize hdfs:// scheme in during AvroFileInputStream initialization.
> There is a unit TestTextCommand.java under hadoop-common project. However it only tested files in local file system. I created a similar one under hadoop-hdfs project using MiniDFSCluster. Please see attached maven unit test error message with full stack trace for more details.
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira