Re: can not get config after launch master process

[Jonathan Hsieh <jo...@cloudera.com>]

Junxian,

Can you do a stack dump to see what is stuck on the master?

Try going to http://master:35871/stacks and if that fails try login and run
'jstacks <pid>'

Thanks,
Jon.

On Thu, Jul 28, 2011 at 6:54 PM, Junxian Yan <ju...@gmail.com> wrote:

> Seems not like this case. Here is part of my configure file
>
>   connect MASTER:35873
>   exec map MASTER collector1
>   exec map MASTER agent1
>
>   exec config collector1 logicalSource
> 'collectorSink("s3n://%{tailSrcFile}/%Y-%m-%d/","%{tailSrcFile}-%Y%m%d.")'
>   exec config agent1
> 'tailDir("/var/apps/log/",fileregex="*.log$",startFromEnd=true)'
> 'logicalSink("collector1")'
>
>
>
>    exec map MASTER collector2
>    exec map MACHINE1 agent2
>
>    exec config collector2 logicalSource
> 'collectorSink("s3n://%{tailSrcFile}/%Y-%m-%d/","%{tailSrcFile}-%Y%m%d.")'
>    exec config agent2
> 'tailDir("/var/apps/log/",fileregex="*.log$",startFromEnd=true)'
> 'logicalSink("collector2")'
>
>
> R
>
> On Fri, Jul 29, 2011 at 9:05 AM, Dan Everton <da...@iocaine.org> wrote:
>
>> Are your configuration scripts calling "exec refreshAll"? We've found
>> that using that command can cause a deadlock in the master process. See
>> this message for an example stack trace:
>>
>>
>> http://mail-archives.apache.org/mod_mbox/incubator-flume-user/201107.mbox/%3C1311734821.16079.2156289349@webmail.messagingengine.com%3E
>>
>> On Thu, 28 Jul 2011 16:49 -0700, "Junxian Yan" <ju...@gmail.com>
>> wrote:
>> > Hi Guys
>> >
>> > I encountered a weird problem. When I start a master process and a
>> couple
>> > of
>> > node process, run the configure script on each node and master machine,
>> I
>> > tried  to access the MASTER:35871. I found I can not get the
>> > configuration.
>> >  Seems the system hang. I also tried it in flume shell. The same
>> problem.
>> > After I send the getconfigs command, no return and the command didn't
>> > complete, no error message. When I restart master process, it's ok.
>> >
>> > I wanna know if anyone else encountered the same issue with me. Is there
>> > any
>> > investigation for this issue. I'm sorry I can not provide flume log, I
>> > can
>> > not reproduce it in short period.
>> >
>> > My flume environment is:   there're 10 collectors and master in the same
>> > machine. And there are another 5 machines running flume node. Every
>> time,
>> > the flume node will connect to master, submit the configuration.
>> >
>> > R
>> >
>>
>
>


-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com