You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Mohit Anchlia <mo...@gmail.com> on 2012/11/13 02:40:48 UTC
Reading files from a directory
Using Java dfs api is it possible to read all the files in a directory? Or
do I need to list all the files in the directory and then read it?
Re: Reading files from a directory
Posted by Harsh J <ha...@cloudera.com>.
Yes you need to list all files, or the files you need, and then read
them in serial or parallel order (per your preference). A "directory"
can't be read on its own.
On Tue, Nov 13, 2012 at 7:10 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> Using Java dfs api is it possible to read all the files in a directory? Or
> do I need to list all the files in the directory and then read it?
--
Harsh J
Re: Reading files from a directory
Posted by Harsh J <ha...@cloudera.com>.
Yes you need to list all files, or the files you need, and then read
them in serial or parallel order (per your preference). A "directory"
can't be read on its own.
On Tue, Nov 13, 2012 at 7:10 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> Using Java dfs api is it possible to read all the files in a directory? Or
> do I need to list all the files in the directory and then read it?
--
Harsh J
Re: Reading files from a directory
Posted by Mohit Anchlia <mo...@gmail.com>.
I was actually looking for an example to do it in the java code. But I
think I've found a way to do it by iterating over all the files using
globStatus() method.
On Mon, Nov 12, 2012 at 5:50 PM, yinghua hu <yi...@gmail.com> wrote:
> Hi, Mohit
>
> You can input everything in a directory. See the step 12 in this link.
>
> http://raseshmori.wordpress.com/
>
>
> On Mon, Nov 12, 2012 at 5:40 PM, Mohit Anchlia <mo...@gmail.com>wrote:
>
>> Using Java dfs api is it possible to read all the files in a directory?
>> Or do I need to list all the files in the directory and then read it?
>
>
>
>
> --
> Regards,
>
> Yinghua
>
Re: Reading files from a directory
Posted by Mohit Anchlia <mo...@gmail.com>.
I was actually looking for an example to do it in the java code. But I
think I've found a way to do it by iterating over all the files using
globStatus() method.
On Mon, Nov 12, 2012 at 5:50 PM, yinghua hu <yi...@gmail.com> wrote:
> Hi, Mohit
>
> You can input everything in a directory. See the step 12 in this link.
>
> http://raseshmori.wordpress.com/
>
>
> On Mon, Nov 12, 2012 at 5:40 PM, Mohit Anchlia <mo...@gmail.com>wrote:
>
>> Using Java dfs api is it possible to read all the files in a directory?
>> Or do I need to list all the files in the directory and then read it?
>
>
>
>
> --
> Regards,
>
> Yinghua
>
Re: Reading files from a directory
Posted by Mohit Anchlia <mo...@gmail.com>.
I was actually looking for an example to do it in the java code. But I
think I've found a way to do it by iterating over all the files using
globStatus() method.
On Mon, Nov 12, 2012 at 5:50 PM, yinghua hu <yi...@gmail.com> wrote:
> Hi, Mohit
>
> You can input everything in a directory. See the step 12 in this link.
>
> http://raseshmori.wordpress.com/
>
>
> On Mon, Nov 12, 2012 at 5:40 PM, Mohit Anchlia <mo...@gmail.com>wrote:
>
>> Using Java dfs api is it possible to read all the files in a directory?
>> Or do I need to list all the files in the directory and then read it?
>
>
>
>
> --
> Regards,
>
> Yinghua
>
Re: Reading files from a directory
Posted by Mohit Anchlia <mo...@gmail.com>.
I was actually looking for an example to do it in the java code. But I
think I've found a way to do it by iterating over all the files using
globStatus() method.
On Mon, Nov 12, 2012 at 5:50 PM, yinghua hu <yi...@gmail.com> wrote:
> Hi, Mohit
>
> You can input everything in a directory. See the step 12 in this link.
>
> http://raseshmori.wordpress.com/
>
>
> On Mon, Nov 12, 2012 at 5:40 PM, Mohit Anchlia <mo...@gmail.com>wrote:
>
>> Using Java dfs api is it possible to read all the files in a directory?
>> Or do I need to list all the files in the directory and then read it?
>
>
>
>
> --
> Regards,
>
> Yinghua
>
Re: Reading files from a directory
Posted by yinghua hu <yi...@gmail.com>.
Hi, Mohit
You can input everything in a directory. See the step 12 in this link.
http://raseshmori.wordpress.com/
On Mon, Nov 12, 2012 at 5:40 PM, Mohit Anchlia <mo...@gmail.com>wrote:
> Using Java dfs api is it possible to read all the files in a directory? Or
> do I need to list all the files in the directory and then read it?
--
Regards,
Yinghua
Re: Reading files from a directory
Posted by Harsh J <ha...@cloudera.com>.
Yes you need to list all files, or the files you need, and then read
them in serial or parallel order (per your preference). A "directory"
can't be read on its own.
On Tue, Nov 13, 2012 at 7:10 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> Using Java dfs api is it possible to read all the files in a directory? Or
> do I need to list all the files in the directory and then read it?
--
Harsh J
Re: Reading files from a directory
Posted by yinghua hu <yi...@gmail.com>.
Hi, Mohit
You can input everything in a directory. See the step 12 in this link.
http://raseshmori.wordpress.com/
On Mon, Nov 12, 2012 at 5:40 PM, Mohit Anchlia <mo...@gmail.com>wrote:
> Using Java dfs api is it possible to read all the files in a directory? Or
> do I need to list all the files in the directory and then read it?
--
Regards,
Yinghua
Re: Reading files from a directory
Posted by Harsh J <ha...@cloudera.com>.
Yes you need to list all files, or the files you need, and then read
them in serial or parallel order (per your preference). A "directory"
can't be read on its own.
On Tue, Nov 13, 2012 at 7:10 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> Using Java dfs api is it possible to read all the files in a directory? Or
> do I need to list all the files in the directory and then read it?
--
Harsh J
Re: Reading files from a directory
Posted by yinghua hu <yi...@gmail.com>.
Hi, Mohit
You can input everything in a directory. See the step 12 in this link.
http://raseshmori.wordpress.com/
On Mon, Nov 12, 2012 at 5:40 PM, Mohit Anchlia <mo...@gmail.com>wrote:
> Using Java dfs api is it possible to read all the files in a directory? Or
> do I need to list all the files in the directory and then read it?
--
Regards,
Yinghua
Re: Reading files from a directory
Posted by yinghua hu <yi...@gmail.com>.
Hi, Mohit
You can input everything in a directory. See the step 12 in this link.
http://raseshmori.wordpress.com/
On Mon, Nov 12, 2012 at 5:40 PM, Mohit Anchlia <mo...@gmail.com>wrote:
> Using Java dfs api is it possible to read all the files in a directory? Or
> do I need to list all the files in the directory and then read it?
--
Regards,
Yinghua