You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Shahab Yunus <sh...@gmail.com> on 2014/07/17 16:58:26 UTC
Providing a file instead of a directory to a M/R job
In MRv2 or Yarn is it possible to provide a complete path to a file instead
of a directory to a mapreduce job?
Usually we provide a list of directory paths by using
FileInputFormat.addInputPath. Can we provide a path which is a full path to
an actually file? I have tried it but getting unexpected permission errors.
Thanks.
Re: Providing a file instead of a directory to a M/R job
Posted by Shahab Yunus <sh...@gmail.com>.
That is what I thought so too but when I give the parent directory as the
input path of that same file, it works. Perhaps I am messing something up.
I am suing cloudera 4.6 btw.
Meanwhile I have noticed that I can read files directly by using
MultileInputs and that works fine.
Regards,
Shahab
On Thu, Jul 17, 2014 at 11:23 AM, Bertrand Dechoux <de...@gmail.com>
wrote:
> No reason why not. And a permission explains why there is an error :
> missing access rights....
>
> Bertrand Dechoux
>
>
> On Thu, Jul 17, 2014 at 4:58 PM, Shahab Yunus <sh...@gmail.com>
> wrote:
>
>> In MRv2 or Yarn is it possible to provide a complete path to a file
>> instead of a directory to a mapreduce job?
>>
>> Usually we provide a list of directory paths by using
>> FileInputFormat.addInputPath. Can we provide a path which is a full path to
>> an actually file? I have tried it but getting unexpected permission errors.
>>
>> Thanks.
>>
>
>
Re: Providing a file instead of a directory to a M/R job
Posted by Shahab Yunus <sh...@gmail.com>.
That is what I thought so too but when I give the parent directory as the
input path of that same file, it works. Perhaps I am messing something up.
I am suing cloudera 4.6 btw.
Meanwhile I have noticed that I can read files directly by using
MultileInputs and that works fine.
Regards,
Shahab
On Thu, Jul 17, 2014 at 11:23 AM, Bertrand Dechoux <de...@gmail.com>
wrote:
> No reason why not. And a permission explains why there is an error :
> missing access rights....
>
> Bertrand Dechoux
>
>
> On Thu, Jul 17, 2014 at 4:58 PM, Shahab Yunus <sh...@gmail.com>
> wrote:
>
>> In MRv2 or Yarn is it possible to provide a complete path to a file
>> instead of a directory to a mapreduce job?
>>
>> Usually we provide a list of directory paths by using
>> FileInputFormat.addInputPath. Can we provide a path which is a full path to
>> an actually file? I have tried it but getting unexpected permission errors.
>>
>> Thanks.
>>
>
>
Re: Providing a file instead of a directory to a M/R job
Posted by Shahab Yunus <sh...@gmail.com>.
That is what I thought so too but when I give the parent directory as the
input path of that same file, it works. Perhaps I am messing something up.
I am suing cloudera 4.6 btw.
Meanwhile I have noticed that I can read files directly by using
MultileInputs and that works fine.
Regards,
Shahab
On Thu, Jul 17, 2014 at 11:23 AM, Bertrand Dechoux <de...@gmail.com>
wrote:
> No reason why not. And a permission explains why there is an error :
> missing access rights....
>
> Bertrand Dechoux
>
>
> On Thu, Jul 17, 2014 at 4:58 PM, Shahab Yunus <sh...@gmail.com>
> wrote:
>
>> In MRv2 or Yarn is it possible to provide a complete path to a file
>> instead of a directory to a mapreduce job?
>>
>> Usually we provide a list of directory paths by using
>> FileInputFormat.addInputPath. Can we provide a path which is a full path to
>> an actually file? I have tried it but getting unexpected permission errors.
>>
>> Thanks.
>>
>
>
Re: Providing a file instead of a directory to a M/R job
Posted by Shahab Yunus <sh...@gmail.com>.
That is what I thought so too but when I give the parent directory as the
input path of that same file, it works. Perhaps I am messing something up.
I am suing cloudera 4.6 btw.
Meanwhile I have noticed that I can read files directly by using
MultileInputs and that works fine.
Regards,
Shahab
On Thu, Jul 17, 2014 at 11:23 AM, Bertrand Dechoux <de...@gmail.com>
wrote:
> No reason why not. And a permission explains why there is an error :
> missing access rights....
>
> Bertrand Dechoux
>
>
> On Thu, Jul 17, 2014 at 4:58 PM, Shahab Yunus <sh...@gmail.com>
> wrote:
>
>> In MRv2 or Yarn is it possible to provide a complete path to a file
>> instead of a directory to a mapreduce job?
>>
>> Usually we provide a list of directory paths by using
>> FileInputFormat.addInputPath. Can we provide a path which is a full path to
>> an actually file? I have tried it but getting unexpected permission errors.
>>
>> Thanks.
>>
>
>
Re: Providing a file instead of a directory to a M/R job
Posted by Bertrand Dechoux <de...@gmail.com>.
No reason why not. And a permission explains why there is an error :
missing access rights....
Bertrand Dechoux
On Thu, Jul 17, 2014 at 4:58 PM, Shahab Yunus <sh...@gmail.com>
wrote:
> In MRv2 or Yarn is it possible to provide a complete path to a file
> instead of a directory to a mapreduce job?
>
> Usually we provide a list of directory paths by using
> FileInputFormat.addInputPath. Can we provide a path which is a full path to
> an actually file? I have tried it but getting unexpected permission errors.
>
> Thanks.
>
Re: Providing a file instead of a directory to a M/R job
Posted by Bertrand Dechoux <de...@gmail.com>.
No reason why not. And a permission explains why there is an error :
missing access rights....
Bertrand Dechoux
On Thu, Jul 17, 2014 at 4:58 PM, Shahab Yunus <sh...@gmail.com>
wrote:
> In MRv2 or Yarn is it possible to provide a complete path to a file
> instead of a directory to a mapreduce job?
>
> Usually we provide a list of directory paths by using
> FileInputFormat.addInputPath. Can we provide a path which is a full path to
> an actually file? I have tried it but getting unexpected permission errors.
>
> Thanks.
>
Re: Providing a file instead of a directory to a M/R job
Posted by Bertrand Dechoux <de...@gmail.com>.
No reason why not. And a permission explains why there is an error :
missing access rights....
Bertrand Dechoux
On Thu, Jul 17, 2014 at 4:58 PM, Shahab Yunus <sh...@gmail.com>
wrote:
> In MRv2 or Yarn is it possible to provide a complete path to a file
> instead of a directory to a mapreduce job?
>
> Usually we provide a list of directory paths by using
> FileInputFormat.addInputPath. Can we provide a path which is a full path to
> an actually file? I have tried it but getting unexpected permission errors.
>
> Thanks.
>
Re: Providing a file instead of a directory to a M/R job
Posted by Bertrand Dechoux <de...@gmail.com>.
No reason why not. And a permission explains why there is an error :
missing access rights....
Bertrand Dechoux
On Thu, Jul 17, 2014 at 4:58 PM, Shahab Yunus <sh...@gmail.com>
wrote:
> In MRv2 or Yarn is it possible to provide a complete path to a file
> instead of a directory to a mapreduce job?
>
> Usually we provide a list of directory paths by using
> FileInputFormat.addInputPath. Can we provide a path which is a full path to
> an actually file? I have tried it but getting unexpected permission errors.
>
> Thanks.
>