You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by David Novogrodsky <da...@gmail.com> on 2014/11/07 01:13:52 UTC
Is there a way to view the files in HDFS directly?
Hello,
Is there a way to view files in the Hadoop HDFS? I have a directory of
files like this:
[cloudera@quickstart ~]$ hdfs dfs -ls /user/hive/warehouse/categories
Found 2 items
-rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
/user/hive/warehouse/categories/_SUCCESS
-rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
/user/hive/warehouse/categories/part-m-00000.avro
[cloudera@quickstart ~]$
[cloudera@quickstart ~]$
I imported a table using Sqoop.
I tried the following commands but they are not working:
--
Found 2 items
-rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
/user/hive/warehouse/categories/_SUCCESS
-rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
/user/hive/warehouse/categories/part-m-00000.avro
[cloudera@quickstart ~]$
[cloudera@quickstart ~]$
[cloudera@quickstart ~]$ hdfs dfs -text
/user/hive/warehouse/categories/_SUCCESS
[cloudera@quickstart ~]$
[cloudera@quickstart ~]$ hdfs dfs -text
/user/hive/warehouse/categories/_SUCCESS
[cloudera@quickstart ~]$
[cloudera@quickstart ~]$ hdfs dfs -text
/user/hive/warehouse/categories/part-m-00000.avro
-text: URI scheme is not "file"
Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
[cloudera@quickstart ~]$ hadoop fs -text
/user/hive/warehouse/categories/part-m-00000.avro
-text: URI scheme is not "file"
Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
[cloudera@quickstart ~]$
David Novogrodsky
david.novogrodsky@gmail.com
http://www.linkedin.com/in/davidnovogrodsky
Re: Is there a way to view the files in HDFS directly?
Posted by Pumudu ruhunage <pu...@gmail.com>.
Hi David,
You can try "cat" command and copy the file to local filesystem and view it[
1 <http://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html#cat>]. But it
seems you need to view it directly within hdfs. i'm not sure if there's any
hadoop commands to get it done, but i have read/write to hdfs using small
java applets. It's well documented in following links [2
<http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample>] [3
<https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-read-a-file-from-hdfs>]
. Hope it helps.
reference
[1] http://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html#cat
[2] http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample
[3]
https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-read-a-file-from-hdfs
Regards,
On 7 November 2014 05:43, David Novogrodsky <da...@gmail.com>
wrote:
> Hello,
>
> Is there a way to view files in the Hadoop HDFS? I have a directory of
> files like this:
> [cloudera@quickstart ~]$ hdfs dfs -ls /user/hive/warehouse/categories
>
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> I imported a table using Sqoop.
>
> I tried the following commands but they are not working:
> --
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$ hadoop fs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$
>
> David Novogrodsky
> david.novogrodsky@gmail.com
> http://www.linkedin.com/in/davidnovogrodsky
>
Re: Is there a way to view the files in HDFS directly?
Posted by Pumudu ruhunage <pu...@gmail.com>.
Hi David,
You can try "cat" command and copy the file to local filesystem and view it[
1 <http://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html#cat>]. But it
seems you need to view it directly within hdfs. i'm not sure if there's any
hadoop commands to get it done, but i have read/write to hdfs using small
java applets. It's well documented in following links [2
<http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample>] [3
<https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-read-a-file-from-hdfs>]
. Hope it helps.
reference
[1] http://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html#cat
[2] http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample
[3]
https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-read-a-file-from-hdfs
Regards,
On 7 November 2014 05:43, David Novogrodsky <da...@gmail.com>
wrote:
> Hello,
>
> Is there a way to view files in the Hadoop HDFS? I have a directory of
> files like this:
> [cloudera@quickstart ~]$ hdfs dfs -ls /user/hive/warehouse/categories
>
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> I imported a table using Sqoop.
>
> I tried the following commands but they are not working:
> --
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$ hadoop fs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$
>
> David Novogrodsky
> david.novogrodsky@gmail.com
> http://www.linkedin.com/in/davidnovogrodsky
>
Re: Is there a way to view the files in HDFS directly?
Posted by Pumudu ruhunage <pu...@gmail.com>.
Hi David,
You can try "cat" command and copy the file to local filesystem and view it[
1 <http://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html#cat>]. But it
seems you need to view it directly within hdfs. i'm not sure if there's any
hadoop commands to get it done, but i have read/write to hdfs using small
java applets. It's well documented in following links [2
<http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample>] [3
<https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-read-a-file-from-hdfs>]
. Hope it helps.
reference
[1] http://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html#cat
[2] http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample
[3]
https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-read-a-file-from-hdfs
Regards,
On 7 November 2014 05:43, David Novogrodsky <da...@gmail.com>
wrote:
> Hello,
>
> Is there a way to view files in the Hadoop HDFS? I have a directory of
> files like this:
> [cloudera@quickstart ~]$ hdfs dfs -ls /user/hive/warehouse/categories
>
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> I imported a table using Sqoop.
>
> I tried the following commands but they are not working:
> --
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$ hadoop fs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$
>
> David Novogrodsky
> david.novogrodsky@gmail.com
> http://www.linkedin.com/in/davidnovogrodsky
>
Re: Is there a way to view the files in HDFS directly?
Posted by "hadoop.support@visolve.com" <ha...@visolve.com>.
Hello,
In normal you can view the file by issuing the command
#hadoop dfs -cat <path-to-file>
Default file store location in /user/<name>/<file-name>
Regards,
ViSolve
On 11/07/2014 05:43 AM, David Novogrodsky wrote:
> Hello,
>
> Is there a way to view files in the Hadoop HDFS? I have a directory
> of files like this:
> [cloudera@quickstart ~]$ hdfs dfs -ls /user/hive/warehouse/categories
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> I imported a table using Sqoop.
>
> I tried the following commands but they are not working:
> --
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$ hadoop fs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$
>
> David Novogrodsky
> david.novogrodsky@gmail.com <ma...@gmail.com>
> http://www.linkedin.com/in/davidnovogrodsky
Re: Is there a way to view the files in HDFS directly?
Posted by "hadoop.support@visolve.com" <ha...@visolve.com>.
Hello,
In normal you can view the file by issuing the command
#hadoop dfs -cat <path-to-file>
Default file store location in /user/<name>/<file-name>
Regards,
ViSolve
On 11/07/2014 05:43 AM, David Novogrodsky wrote:
> Hello,
>
> Is there a way to view files in the Hadoop HDFS? I have a directory
> of files like this:
> [cloudera@quickstart ~]$ hdfs dfs -ls /user/hive/warehouse/categories
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> I imported a table using Sqoop.
>
> I tried the following commands but they are not working:
> --
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$ hadoop fs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$
>
> David Novogrodsky
> david.novogrodsky@gmail.com <ma...@gmail.com>
> http://www.linkedin.com/in/davidnovogrodsky
Re: Is there a way to view the files in HDFS directly?
Posted by "hadoop.support@visolve.com" <ha...@visolve.com>.
Hello,
In normal you can view the file by issuing the command
#hadoop dfs -cat <path-to-file>
Default file store location in /user/<name>/<file-name>
Regards,
ViSolve
On 11/07/2014 05:43 AM, David Novogrodsky wrote:
> Hello,
>
> Is there a way to view files in the Hadoop HDFS? I have a directory
> of files like this:
> [cloudera@quickstart ~]$ hdfs dfs -ls /user/hive/warehouse/categories
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> I imported a table using Sqoop.
>
> I tried the following commands but they are not working:
> --
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$ hadoop fs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$
>
> David Novogrodsky
> david.novogrodsky@gmail.com <ma...@gmail.com>
> http://www.linkedin.com/in/davidnovogrodsky
Re: Is there a way to view the files in HDFS directly?
Posted by "hadoop.support@visolve.com" <ha...@visolve.com>.
Hello,
In normal you can view the file by issuing the command
#hadoop dfs -cat <path-to-file>
Default file store location in /user/<name>/<file-name>
Regards,
ViSolve
On 11/07/2014 05:43 AM, David Novogrodsky wrote:
> Hello,
>
> Is there a way to view files in the Hadoop HDFS? I have a directory
> of files like this:
> [cloudera@quickstart ~]$ hdfs dfs -ls /user/hive/warehouse/categories
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> I imported a table using Sqoop.
>
> I tried the following commands but they are not working:
> --
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$ hadoop fs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$
>
> David Novogrodsky
> david.novogrodsky@gmail.com <ma...@gmail.com>
> http://www.linkedin.com/in/davidnovogrodsky
Re: Is there a way to view the files in HDFS directly?
Posted by Pumudu ruhunage <pu...@gmail.com>.
Hi David,
You can try "cat" command and copy the file to local filesystem and view it[
1 <http://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html#cat>]. But it
seems you need to view it directly within hdfs. i'm not sure if there's any
hadoop commands to get it done, but i have read/write to hdfs using small
java applets. It's well documented in following links [2
<http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample>] [3
<https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-read-a-file-from-hdfs>]
. Hope it helps.
reference
[1] http://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html#cat
[2] http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample
[3]
https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-read-a-file-from-hdfs
Regards,
On 7 November 2014 05:43, David Novogrodsky <da...@gmail.com>
wrote:
> Hello,
>
> Is there a way to view files in the Hadoop HDFS? I have a directory of
> files like this:
> [cloudera@quickstart ~]$ hdfs dfs -ls /user/hive/warehouse/categories
>
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> I imported a table using Sqoop.
>
> I tried the following commands but they are not working:
> --
> Found 2 items
> -rw-r--r-- 1 cloudera hive 0 2014-11-05 13:18
> /user/hive/warehouse/categories/_SUCCESS
> -rw-r--r-- 1 cloudera hive 1344 2014-11-05 13:18
> /user/hive/warehouse/categories/part-m-00000.avro
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/_SUCCESS
> [cloudera@quickstart ~]$
> [cloudera@quickstart ~]$ hdfs dfs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$ hadoop fs -text
> /user/hive/warehouse/categories/part-m-00000.avro
> -text: URI scheme is not "file"
> Usage: hadoop fs [generic options] -text [-ignoreCrc] <src> ...
> [cloudera@quickstart ~]$
>
> David Novogrodsky
> david.novogrodsky@gmail.com
> http://www.linkedin.com/in/davidnovogrodsky
>