You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hadoop.apache.org by Priyanka Gugale <pr...@apache.org> on 2017/04/17 10:06:45 UTC

[HDFS] Monitor a directory

Hi,

I would like to monitor a HDFS directory for any file
addition/deletion/modification.
I see we have a inotify at file system level. I see two problems with it,
1. It has to be run as a DFSAdmin user.
2. I didn't find a way to monitor a particular folder, it monitors entire
file system.

Is there any other way I can monitor a directory? Or is there something I
am missing about inotify?

-Priyanka

Re: [HDFS] Monitor a directory

Posted by priyanka gugale <pr...@apache.org>.

Okay got it. Thanks for your inputs.

-Priyanka

On Mon, Apr 17, 2017 at 7:47 PM, surendra lilhore <
surendra.lilhore@huawei.com> wrote:

> Hi Priyanka,
>
>
>
> Currently Inotify is only one option available in HDFS to monitor the file
> system. It is not supporting path based monitoring. Some work is
> in-progress to support it.
>
>
>
> https://issues.apache.org/jira/browse/HDFS-6939
>
> https://issues.apache.org/jira/browse/HDFS-8940
>
>
>
> Regards
>
> -Surendra
>
>
>
> *From:* Priyanka Gugale [mailto:priyag@apache.org]
> *Sent:* 17 April 2017 15:37
> *To:* user@hadoop.apache.org
> *Subject:* [HDFS] Monitor a directory
>
>
>
> Hi,
>
>
>
> I would like to monitor a HDFS directory for any file addition/deletion/
> modification.
> I see we have a inotify at file system level. I see two problems with it,
>
> 1. It has to be run as a DFSAdmin user.
>
> 2. I didn't find a way to monitor a particular folder, it monitors entire
> file system.
>
>
>
> Is there any other way I can monitor a directory? Or is there something I
> am missing about inotify?
>
>
>
> -Priyanka
>
>
>

RE: [HDFS] Monitor a directory

Posted by surendra lilhore <su...@huawei.com>.

Hi Priyanka,

Currently Inotify is only one option available in HDFS to monitor the file system. It is not supporting path based monitoring. Some work is in-progress to support it.

https://issues.apache.org/jira/browse/HDFS-6939
https://issues.apache.org/jira/browse/HDFS-8940

Regards
-Surendra

From: Priyanka Gugale [mailto:priyag@apache.org]
Sent: 17 April 2017 15:37
To: user@hadoop.apache.org
Subject: [HDFS] Monitor a directory

Hi,

I would like to monitor a HDFS directory for any file addition/deletion/modification.
I see we have a inotify at file system level. I see two problems with it,
1. It has to be run as a DFSAdmin user.
2. I didn't find a way to monitor a particular folder, it monitors entire file system.

Is there any other way I can monitor a directory? Or is there something I am missing about inotify?

-Priyanka