You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jay Vyas <ja...@gmail.com> on 2013/01/14 22:24:29 UTC
FileSystem.workingDir vs mapred.local.dir
Hi guys: What is the relationship between the "working directory" in the
FileSystem class (filesystem.workingDir), compared with the
mapred.local.dir properties ?
It seems like these would essentially refer to the same thing?
--
Jay Vyas
http://jayunit100.blogspot.com
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Jay Vyas <ja...@gmail.com>.
ah okay. so - in default hadoop dfs, the workingDir is (i beleive)
/user/hadoop/ , because as i recall when putting a file into hdfs, that
seems to be where the files naturally end up if there is no path
specified.
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Jay Vyas <ja...@gmail.com>.
ah okay. so - in default hadoop dfs, the workingDir is (i beleive)
/user/hadoop/ , because as i recall when putting a file into hdfs, that
seems to be where the files naturally end up if there is no path
specified.
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Jay Vyas <ja...@gmail.com>.
ah okay. so - in default hadoop dfs, the workingDir is (i beleive)
/user/hadoop/ , because as i recall when putting a file into hdfs, that
seems to be where the files naturally end up if there is no path
specified.
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Jay Vyas <ja...@gmail.com>.
ah okay. so - in default hadoop dfs, the workingDir is (i beleive)
/user/hadoop/ , because as i recall when putting a file into hdfs, that
seems to be where the files naturally end up if there is no path
specified.
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Harsh J <ha...@cloudera.com>.
Jay,
For your FS question: http://en.wikipedia.org/wiki/Working_directory
On Wed, Jan 16, 2013 at 12:17 AM, Jay Vyas <ja...@gmail.com> wrote:
> what do you mean by "workingdir for a filesystem" ? I never thought that
> a fileSystem should
> need or have a special workingDir ?
>
>
> On Tue, Jan 15, 2013 at 12:43 PM, Hemanth Yamijala <
> yhemanth@thoughtworks.com> wrote:
>
>> Hi,
>>
>> AFAIK, the mapred.local.dir property refers to a set of directories under
>> which different types of data related to mapreduce jobs are stored - for
>> e.g. intermediate data, localized files for a job etc. The working
>> directory for a mapreduce job is configured under a sub directory within
>> one of the directories configured in this property.
>>
>> The workingDir property of the FileSystem class simply seems to indicate
>> the working directory for a given filesystem as set by applications.
>>
>> They don't seem very related per se, unless I am missing something ?
>>
>> Thanks
>> Hemanth
>>
>>
>> On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
>>
>>> Hi guys: What is the relationship between the "working directory" in
>>> the FileSystem class (filesystem.workingDir), compared with the
>>> mapred.local.dir properties ?
>>>
>>> It seems like these would essentially refer to the same thing?
>>> --
>>> Jay Vyas
>>> http://jayunit100.blogspot.com
>>>
>>
>>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
Harsh J
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Harsh J <ha...@cloudera.com>.
Jay,
For your FS question: http://en.wikipedia.org/wiki/Working_directory
On Wed, Jan 16, 2013 at 12:17 AM, Jay Vyas <ja...@gmail.com> wrote:
> what do you mean by "workingdir for a filesystem" ? I never thought that
> a fileSystem should
> need or have a special workingDir ?
>
>
> On Tue, Jan 15, 2013 at 12:43 PM, Hemanth Yamijala <
> yhemanth@thoughtworks.com> wrote:
>
>> Hi,
>>
>> AFAIK, the mapred.local.dir property refers to a set of directories under
>> which different types of data related to mapreduce jobs are stored - for
>> e.g. intermediate data, localized files for a job etc. The working
>> directory for a mapreduce job is configured under a sub directory within
>> one of the directories configured in this property.
>>
>> The workingDir property of the FileSystem class simply seems to indicate
>> the working directory for a given filesystem as set by applications.
>>
>> They don't seem very related per se, unless I am missing something ?
>>
>> Thanks
>> Hemanth
>>
>>
>> On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
>>
>>> Hi guys: What is the relationship between the "working directory" in
>>> the FileSystem class (filesystem.workingDir), compared with the
>>> mapred.local.dir properties ?
>>>
>>> It seems like these would essentially refer to the same thing?
>>> --
>>> Jay Vyas
>>> http://jayunit100.blogspot.com
>>>
>>
>>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
Harsh J
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Harsh J <ha...@cloudera.com>.
Jay,
For your FS question: http://en.wikipedia.org/wiki/Working_directory
On Wed, Jan 16, 2013 at 12:17 AM, Jay Vyas <ja...@gmail.com> wrote:
> what do you mean by "workingdir for a filesystem" ? I never thought that
> a fileSystem should
> need or have a special workingDir ?
>
>
> On Tue, Jan 15, 2013 at 12:43 PM, Hemanth Yamijala <
> yhemanth@thoughtworks.com> wrote:
>
>> Hi,
>>
>> AFAIK, the mapred.local.dir property refers to a set of directories under
>> which different types of data related to mapreduce jobs are stored - for
>> e.g. intermediate data, localized files for a job etc. The working
>> directory for a mapreduce job is configured under a sub directory within
>> one of the directories configured in this property.
>>
>> The workingDir property of the FileSystem class simply seems to indicate
>> the working directory for a given filesystem as set by applications.
>>
>> They don't seem very related per se, unless I am missing something ?
>>
>> Thanks
>> Hemanth
>>
>>
>> On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
>>
>>> Hi guys: What is the relationship between the "working directory" in
>>> the FileSystem class (filesystem.workingDir), compared with the
>>> mapred.local.dir properties ?
>>>
>>> It seems like these would essentially refer to the same thing?
>>> --
>>> Jay Vyas
>>> http://jayunit100.blogspot.com
>>>
>>
>>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
Harsh J
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Harsh J <ha...@cloudera.com>.
Jay,
For your FS question: http://en.wikipedia.org/wiki/Working_directory
On Wed, Jan 16, 2013 at 12:17 AM, Jay Vyas <ja...@gmail.com> wrote:
> what do you mean by "workingdir for a filesystem" ? I never thought that
> a fileSystem should
> need or have a special workingDir ?
>
>
> On Tue, Jan 15, 2013 at 12:43 PM, Hemanth Yamijala <
> yhemanth@thoughtworks.com> wrote:
>
>> Hi,
>>
>> AFAIK, the mapred.local.dir property refers to a set of directories under
>> which different types of data related to mapreduce jobs are stored - for
>> e.g. intermediate data, localized files for a job etc. The working
>> directory for a mapreduce job is configured under a sub directory within
>> one of the directories configured in this property.
>>
>> The workingDir property of the FileSystem class simply seems to indicate
>> the working directory for a given filesystem as set by applications.
>>
>> They don't seem very related per se, unless I am missing something ?
>>
>> Thanks
>> Hemanth
>>
>>
>> On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
>>
>>> Hi guys: What is the relationship between the "working directory" in
>>> the FileSystem class (filesystem.workingDir), compared with the
>>> mapred.local.dir properties ?
>>>
>>> It seems like these would essentially refer to the same thing?
>>> --
>>> Jay Vyas
>>> http://jayunit100.blogspot.com
>>>
>>
>>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
Harsh J
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Jay Vyas <ja...@gmail.com>.
what do you mean by "workingdir for a filesystem" ? I never thought that a
fileSystem should
need or have a special workingDir ?
On Tue, Jan 15, 2013 at 12:43 PM, Hemanth Yamijala <
yhemanth@thoughtworks.com> wrote:
> Hi,
>
> AFAIK, the mapred.local.dir property refers to a set of directories under
> which different types of data related to mapreduce jobs are stored - for
> e.g. intermediate data, localized files for a job etc. The working
> directory for a mapreduce job is configured under a sub directory within
> one of the directories configured in this property.
>
> The workingDir property of the FileSystem class simply seems to indicate
> the working directory for a given filesystem as set by applications.
>
> They don't seem very related per se, unless I am missing something ?
>
> Thanks
> Hemanth
>
>
> On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
>
>> Hi guys: What is the relationship between the "working directory" in the
>> FileSystem class (filesystem.workingDir), compared with the
>> mapred.local.dir properties ?
>>
>> It seems like these would essentially refer to the same thing?
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Jay Vyas <ja...@gmail.com>.
what do you mean by "workingdir for a filesystem" ? I never thought that a
fileSystem should
need or have a special workingDir ?
On Tue, Jan 15, 2013 at 12:43 PM, Hemanth Yamijala <
yhemanth@thoughtworks.com> wrote:
> Hi,
>
> AFAIK, the mapred.local.dir property refers to a set of directories under
> which different types of data related to mapreduce jobs are stored - for
> e.g. intermediate data, localized files for a job etc. The working
> directory for a mapreduce job is configured under a sub directory within
> one of the directories configured in this property.
>
> The workingDir property of the FileSystem class simply seems to indicate
> the working directory for a given filesystem as set by applications.
>
> They don't seem very related per se, unless I am missing something ?
>
> Thanks
> Hemanth
>
>
> On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
>
>> Hi guys: What is the relationship between the "working directory" in the
>> FileSystem class (filesystem.workingDir), compared with the
>> mapred.local.dir properties ?
>>
>> It seems like these would essentially refer to the same thing?
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Jay Vyas <ja...@gmail.com>.
what do you mean by "workingdir for a filesystem" ? I never thought that a
fileSystem should
need or have a special workingDir ?
On Tue, Jan 15, 2013 at 12:43 PM, Hemanth Yamijala <
yhemanth@thoughtworks.com> wrote:
> Hi,
>
> AFAIK, the mapred.local.dir property refers to a set of directories under
> which different types of data related to mapreduce jobs are stored - for
> e.g. intermediate data, localized files for a job etc. The working
> directory for a mapreduce job is configured under a sub directory within
> one of the directories configured in this property.
>
> The workingDir property of the FileSystem class simply seems to indicate
> the working directory for a given filesystem as set by applications.
>
> They don't seem very related per se, unless I am missing something ?
>
> Thanks
> Hemanth
>
>
> On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
>
>> Hi guys: What is the relationship between the "working directory" in the
>> FileSystem class (filesystem.workingDir), compared with the
>> mapred.local.dir properties ?
>>
>> It seems like these would essentially refer to the same thing?
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Jay Vyas <ja...@gmail.com>.
what do you mean by "workingdir for a filesystem" ? I never thought that a
fileSystem should
need or have a special workingDir ?
On Tue, Jan 15, 2013 at 12:43 PM, Hemanth Yamijala <
yhemanth@thoughtworks.com> wrote:
> Hi,
>
> AFAIK, the mapred.local.dir property refers to a set of directories under
> which different types of data related to mapreduce jobs are stored - for
> e.g. intermediate data, localized files for a job etc. The working
> directory for a mapreduce job is configured under a sub directory within
> one of the directories configured in this property.
>
> The workingDir property of the FileSystem class simply seems to indicate
> the working directory for a given filesystem as set by applications.
>
> They don't seem very related per se, unless I am missing something ?
>
> Thanks
> Hemanth
>
>
> On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
>
>> Hi guys: What is the relationship between the "working directory" in the
>> FileSystem class (filesystem.workingDir), compared with the
>> mapred.local.dir properties ?
>>
>> It seems like these would essentially refer to the same thing?
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
Hi,
AFAIK, the mapred.local.dir property refers to a set of directories under
which different types of data related to mapreduce jobs are stored - for
e.g. intermediate data, localized files for a job etc. The working
directory for a mapreduce job is configured under a sub directory within
one of the directories configured in this property.
The workingDir property of the FileSystem class simply seems to indicate
the working directory for a given filesystem as set by applications.
They don't seem very related per se, unless I am missing something ?
Thanks
Hemanth
On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
> Hi guys: What is the relationship between the "working directory" in the
> FileSystem class (filesystem.workingDir), compared with the
> mapred.local.dir properties ?
>
> It seems like these would essentially refer to the same thing?
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
Hi,
AFAIK, the mapred.local.dir property refers to a set of directories under
which different types of data related to mapreduce jobs are stored - for
e.g. intermediate data, localized files for a job etc. The working
directory for a mapreduce job is configured under a sub directory within
one of the directories configured in this property.
The workingDir property of the FileSystem class simply seems to indicate
the working directory for a given filesystem as set by applications.
They don't seem very related per se, unless I am missing something ?
Thanks
Hemanth
On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
> Hi guys: What is the relationship between the "working directory" in the
> FileSystem class (filesystem.workingDir), compared with the
> mapred.local.dir properties ?
>
> It seems like these would essentially refer to the same thing?
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
Hi,
AFAIK, the mapred.local.dir property refers to a set of directories under
which different types of data related to mapreduce jobs are stored - for
e.g. intermediate data, localized files for a job etc. The working
directory for a mapreduce job is configured under a sub directory within
one of the directories configured in this property.
The workingDir property of the FileSystem class simply seems to indicate
the working directory for a given filesystem as set by applications.
They don't seem very related per se, unless I am missing something ?
Thanks
Hemanth
On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
> Hi guys: What is the relationship between the "working directory" in the
> FileSystem class (filesystem.workingDir), compared with the
> mapred.local.dir properties ?
>
> It seems like these would essentially refer to the same thing?
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
Re: FileSystem.workingDir vs mapred.local.dir
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
Hi,
AFAIK, the mapred.local.dir property refers to a set of directories under
which different types of data related to mapreduce jobs are stored - for
e.g. intermediate data, localized files for a job etc. The working
directory for a mapreduce job is configured under a sub directory within
one of the directories configured in this property.
The workingDir property of the FileSystem class simply seems to indicate
the working directory for a given filesystem as set by applications.
They don't seem very related per se, unless I am missing something ?
Thanks
Hemanth
On Tue, Jan 15, 2013 at 2:54 AM, Jay Vyas <ja...@gmail.com> wrote:
> Hi guys: What is the relationship between the "working directory" in the
> FileSystem class (filesystem.workingDir), compared with the
> mapred.local.dir properties ?
>
> It seems like these would essentially refer to the same thing?
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>