flume agent not work

["hao.wang" <ha...@ipinyou.com>]

Hi,All:
    I have a problem about flume. In our production environment, We use flume to transfer logs from web servers to HDFS. We have 3 flume agents. But sometimes, only 1 agent works, the others can not work. I checked the status of flume agents. They are all alived. Does anybody know why?

regards

2011-10-08 



hao.wang 

Re: flume agent not work

[唐亮 <le...@gmail.com>]

��

2011/10/8 hao.wang <ha...@ipinyou.com>

> **
> Hi,All:
>     I have a problem about flume. In our production environment, We use
> flume to transfer logs from web servers to HDFS. We have 3 flume agents. But
> sometimes, only 1 agent works, the others can not work. I checked the status
> of flume agents. They are all alived. Does anybody know why?
>
> regards
>
> 2011-10-08
> ------------------------------
> hao.wang
>

Re: flume agent not work

[Mingjie Lai <mj...@gmail.com>]

Hao.

You may need to provide more infor, e.g, your configurations, 
environment details, and log files, so people can have a better 
understanding the problem.

-mingjie

On 10/08/2011 12:51 AM, hao.wang wrote:
> Hi,All:
> I have a problem about flume. In our production environment, We use
> flume to transfer logs from web servers to HDFS. We have 3 flume agents.
> But sometimes, only 1 agent works, the others can not work. I checked
> the status of flume agents. They are all alived. Does anybody know why?
> regards
> 2011-10-08
> ------------------------------------------------------------------------
> hao.wang

Re: flume agent not work

[Michael Jiang <it...@gmail.com>]

Hao,

First of all, what do you mean all agents are alive. Are their daemons
running or blocked or sleeping? For example, if a daemon is too busy with
GC, it may not serve any function at all.

Hope this already fixed. But in general, I guess there are couple of places
to look at for clues.

1st, there is a web interface you can check status of flume services,
including agents.

2nd, check agent log for clues. Also check collectors so that together will
provide a complete view of possible problems.

3rd, check master for configuration. I guess its low probability that
configuration got altered unexpectedly. But check it as a routine wont hurt
nobody :)

4th, check network connection between agent and collector, and between
collector and hdfs. This may include both hardware, flume configuration and
system network configurations (e.g. any firewall or dns update recently?).

5th, check os log for abnormalities.

By all these means, you may want to reduce problem to a small area, e.g., an
agent may send data to collectors, but it is a collector that fails to relay
the data to hdfs. etc.

Hope this helps.

-mJ

On Sat, Oct 8, 2011 at 12:51 AM, hao.wang <ha...@ipinyou.com> wrote:

> **
> Hi,All:
>     I have a problem about flume. In our production environment, We use
> flume to transfer logs from web servers to HDFS. We have 3 flume agents. But
> sometimes, only 1 agent works, the others can not work. I checked the status
> of flume agents. They are all alived. Does anybody know why?
>
> regards
>
> 2011-10-08
> ------------------------------
> hao.wang
>