File Busy or not check. Kindly Help

[Ravi Shankar <ra...@hcl.com>]

Hi All,

I am trying to check that if a file is busy ,i.e. it is still being copied into DFS from Local, the MR Job should not run if it is, otherwise it should. What I have observed is that if the file is still being written to DFS, and I call the MR job on it, the job runs (which may include operations to move the file, and it is successful), after the completion of the MR Job, if the file is missing, i.e it has been moved by the MR job, the file copying operation throws an exception of "file does not exist".

I need the MR job to wait until the file is completely copied and then the MR job should run on it.

Kindly help on this.

Thanks & Regards,
Ravi Shankar,
Cross Functional Services - Solution- Cloud
HCL Technologies Ltd. - Infrastructure Services Division
F 8,9 Sec - III, Noida, India - 201301. Cell - +91 9953696666


________________________________


::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or contain viruses.The e mail and its contents (with or without referred
errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Any views or opinions
presented in this email are solely those of the author and may not necessarily reflect the opinions of HCL or its
affiliates. Any form of reproduction, dissemination, copying, disclosure, Modification, distribution and/or publication
of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening any mail and attachments
please check them for viruses and defect.

-----------------------------------------------------------------------------------------------------------------------

Re: File Busy or not check. Kindly Help

[Harsh J <ha...@cloudera.com>]

Hey Ravi,

I just answered the same question not too long ago, which you can read
at: http://search-hadoop.com/m/o2DhNQPv7G1

On Tue, May 1, 2012 at 11:39 AM, Ravi Shankar <ra...@hcl.com> wrote:
> Hi All,
>
> I am trying to check that if a file is busy ,i.e. it is still being copied into DFS from Local, the MR Job should not run if it is, otherwise it should. What I have observed is that if the file is still being written to DFS, and I call the MR job on it, the job runs (which may include operations to move the file, and it is successful), after the completion of the MR Job, if the file is missing, i.e it has been moved by the MR job, the file copying operation throws an exception of "file does not exist".
>
> I need the MR job to wait until the file is completely copied and then the MR job should run on it.
>
> Kindly help on this.
>
> Thanks & Regards,
> Ravi Shankar,
> Cross Functional Services - Solution- Cloud
> HCL Technologies Ltd. - Infrastructure Services Division
> F 8,9 Sec - III, Noida, India - 201301. Cell - +91 9953696666
>
>
> ________________________________
>
>
> ::DISCLAIMER::
> -----------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
> E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted,
> lost, destroyed, arrive late or incomplete, or contain viruses.The e mail and its contents (with or without referred
> errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Any views or opinions
> presented in this email are solely those of the author and may not necessarily reflect the opinions of HCL or its
> affiliates. Any form of reproduction, dissemination, copying, disclosure, Modification, distribution and/or publication
> of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
> received this email in error please delete it and notify the sender immediately. Before opening any mail and attachments
> please check them for viruses and defect.
>
> -----------------------------------------------------------------------------------------------------------------------



-- 
Harsh J