Why is FileSystem.createNonRecursive deprecated?

[Ravi Prakash <ra...@ymail.com>]

Hi folks,

I am trying to fix MAPREDUCE-5317. I noticed that the only way through FileSystem to NOT recursively create directories is through the deprecated method 

@deprecated API only for 0.20-append
FileSystem.createNonRecursive.


This has been marked deprecated ever since it was put in by HADOOP-6840. Do we know if we ever expect to un-deprecate this method? I am trying to find the rationale behind checking it in as a deprecated method, but haven't been able to find any written record. Does anyone know?
Thanks
Ravi

Re: Why is FileSystem.createNonRecursive deprecated?

[Kihwal Lee <ki...@yahoo-inc.com>]

It seems the transition within hadoop needs to happen before deprecating
FileSystem. I think createNonRecursive() was marked deprecated from the
beginning under the assumption that this transition would happen soon. We
know that hasn't been the case. So the question is, do we transition the
portion of code (whatever is affecting MAPREDUCE-5317) now or later.

Fixing this bug by calling createNonRecursive() won't add any more work
for the FileContext transition and the component is already depending on
FileSystem. I.e. no new dependency that affects the transition.

During the full transition effort, there will likely be common concerns
that need to be addressed and may even involve code changes in FileContext
or changes in the way callers work. If the conversion of the code as part
of fixing MAPREDUCE-5317 is straightforward and won't be blocked by such
issues, I would say go with FileContext.  Otherwise, I suggest do a quick
fix for now.

Others thoughts?

Kihwal

On 6/12/13 7:58 PM, "Colin McCabe" <cm...@alumni.cmu.edu> wrote:

>This seems inconsistent. If the method is deprecated just because it's
>in org.apache.hadoop.FileSystem, shouldn't all FileSystem methods be
>marked as deprecated?
>
>On the other hand, a user opening up FileSystem.java would probably
>not realize that it is deprecated.  The JavaDoc for the class itself
>doesn't mention it, although the JavaDoc for a few of the methods
>talks about a "transition".
>
>It might make more sense to mark the class as a whole @deprecated if
>that is the intent.
>
>I did a quick search on JIRA, but didn't see anything answering these
>questions.  Opinions?
>
>best,
>Colin
>
>
>On Tue, Jun 11, 2013 at 4:09 PM, Andrew Wang <an...@cloudera.com>
>wrote:
>> Hi Ravi,
>>
>> I wasn't around for HADOOP-6840, but I'm guessing it's deprecated for
>>the
>> same reasons as primitiveCreate: FileSystem is supposed to eventually
>>to be
>> supplanted by FileContext.
>>
>> FileContext#create also has a more manageable number of method
>>signatures
>> through the use of flags, and in fact defaults to not creating parent
>> directories. I believe MR2 also uses FileContext over FileSystem, so
>>this
>> might be your best bet.
>>
>> HTH,
>> Andrew
>>
>>
>> On Tue, Jun 11, 2013 at 3:18 PM, Ravi Prakash <ra...@ymail.com> wrote:
>>
>>> Hi folks,
>>>
>>> I am trying to fix MAPREDUCE-5317. I noticed that the only way through
>>> FileSystem to NOT recursively create directories is through the
>>>deprecated
>>> method
>>>
>>> @deprecated API only for 0.20-append
>>> FileSystem.createNonRecursive.
>>>
>>>
>>> This has been marked deprecated ever since it was put in by
>>>HADOOP-6840.
>>> Do we know if we ever expect to un-deprecate this method? I am trying
>>>to
>>> find the rationale behind checking it in as a deprecated method, but
>>> haven't been able to find any written record. Does anyone know?
>>> Thanks
>>> Ravi

Re: Why is FileSystem.createNonRecursive deprecated?

[Colin McCabe <cm...@alumni.cmu.edu>]

This seems inconsistent. If the method is deprecated just because it's
in org.apache.hadoop.FileSystem, shouldn't all FileSystem methods be
marked as deprecated?

On the other hand, a user opening up FileSystem.java would probably
not realize that it is deprecated.  The JavaDoc for the class itself
doesn't mention it, although the JavaDoc for a few of the methods
talks about a "transition".

It might make more sense to mark the class as a whole @deprecated if
that is the intent.

I did a quick search on JIRA, but didn't see anything answering these
questions.  Opinions?

best,
Colin


On Tue, Jun 11, 2013 at 4:09 PM, Andrew Wang <an...@cloudera.com> wrote:
> Hi Ravi,
>
> I wasn't around for HADOOP-6840, but I'm guessing it's deprecated for the
> same reasons as primitiveCreate: FileSystem is supposed to eventually to be
> supplanted by FileContext.
>
> FileContext#create also has a more manageable number of method signatures
> through the use of flags, and in fact defaults to not creating parent
> directories. I believe MR2 also uses FileContext over FileSystem, so this
> might be your best bet.
>
> HTH,
> Andrew
>
>
> On Tue, Jun 11, 2013 at 3:18 PM, Ravi Prakash <ra...@ymail.com> wrote:
>
>> Hi folks,
>>
>> I am trying to fix MAPREDUCE-5317. I noticed that the only way through
>> FileSystem to NOT recursively create directories is through the deprecated
>> method
>>
>> @deprecated API only for 0.20-append
>> FileSystem.createNonRecursive.
>>
>>
>> This has been marked deprecated ever since it was put in by HADOOP-6840.
>> Do we know if we ever expect to un-deprecate this method? I am trying to
>> find the rationale behind checking it in as a deprecated method, but
>> haven't been able to find any written record. Does anyone know?
>> Thanks
>> Ravi

Re: Why is FileSystem.createNonRecursive deprecated?

[Andrew Wang <an...@cloudera.com>]

Hi Ravi,

I wasn't around for HADOOP-6840, but I'm guessing it's deprecated for the
same reasons as primitiveCreate: FileSystem is supposed to eventually to be
supplanted by FileContext.

FileContext#create also has a more manageable number of method signatures
through the use of flags, and in fact defaults to not creating parent
directories. I believe MR2 also uses FileContext over FileSystem, so this
might be your best bet.

HTH,
Andrew


On Tue, Jun 11, 2013 at 3:18 PM, Ravi Prakash <ra...@ymail.com> wrote:

> Hi folks,
>
> I am trying to fix MAPREDUCE-5317. I noticed that the only way through
> FileSystem to NOT recursively create directories is through the deprecated
> method
>
> @deprecated API only for 0.20-append
> FileSystem.createNonRecursive.
>
>
> This has been marked deprecated ever since it was put in by HADOOP-6840.
> Do we know if we ever expect to un-deprecate this method? I am trying to
> find the rationale behind checking it in as a deprecated method, but
> haven't been able to find any written record. Does anyone know?
> Thanks
> Ravi