You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by 周梦想 <ab...@gmail.com> on 2013/01/23 10:21:35 UTC
ERROR connector.DirectDriver: Exiting driver logicalNode agent2-18 in
error state TailSource | fail because Attempting to open a stub sink 'fail'!
hello,
I add logical config to flume:
agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
collect1: logicalSource |
collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);
map Hadoop46 agent1
map Hadoop47 agent2
map Hadoop48 collect1
============================
I run master on Hadoop48,
it reports:
2013-01-23 16:49:27,153 [main] INFO rolling.RollSink: Created RollSink:
trigger=[TimeTrigger: maxAge=5000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@7a960b08]
checkPeriodMs = 250
spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:27,158 [main] INFO rolling.RollSink: Created RollSink:
trigger=[TimeTrigger: maxAge=5000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1d29109e]
checkPeriodMs = 250
spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:27,207 [main] INFO master.ZooKeeperConfigStore: Created
new config at /flume-cfgs/cfg-0000000104
2013-01-23 16:49:27,207 [main-EventThread] INFO
master.ZooKeeperConfigStore: Config was updated - reloading
2013-01-23 16:49:27,255 [main] INFO master.ZooKeeperConfigStore: Created
new config at /flume-cfgs/cfg-0000000105
2013-01-23 16:49:27,255 [main-EventThread] INFO
master.ZooKeeperConfigStore: Config was updated - reloading
2013-01-23 16:49:27,307 [main] WARN logical.LogicalConfigurationManager:
Logical node 'collect1' has not been assigned to a physical node yet
=================
run agent1 on Hadoop46, it reports:
2013-01-23 16:49:45,070 [Heartbeat] INFO rolling.RollSink: Created
RollSink: trigger=[TimeTrigger: maxAge=5000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1fd54f77]
checkPeriodMs = 250
spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:45,073 [Heartbeat] INFO agent.LogicalNodeManager: creating
new logical node collect1
2013-01-23 16:49:45,076 [logicalNode collect1-18] ERROR
connector.DirectDriver: Closing down due to exception on open calls
2013-01-23 16:49:45,076 [logicalNode collect1-18] INFO
connector.DirectDriver: Connector logicalNode collect1-18 exited with
error: Attempting to open a Stub Source!
2013-01-23 16:49:45,077 [logicalNode collect1-18] ERROR
connector.DirectDriver: Error closing logicalNode collect1-18 source:
Attempting to close a Stub Source!
2013-01-23 16:49:45,077 [logicalNode collect1-18] INFO rolling.RollSink:
closing RollSink
'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:45,081 [logicalNode collect1-18] INFO rolling.RollSink:
double close
'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:45,081 [logicalNode collect1-18] ERROR
connector.DirectDriver: Exiting driver logicalNode collect1-18 in error
state StubSource | Collector because Attempting to open a Stub Source!
2013-01-23 16:49:45,091 [Check config] INFO rolling.RollSink: Created
RollSink: trigger=[TimeTrigger: maxAge=5000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@5780208f]
checkPeriodMs = 250
spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
===============
and agent2 on Hadoop47 , also logs error.
[zhouhh@Hadoop47 ~]$ flume node
2013-01-23 17:11:38,454 [main] INFO mortbay.log: Started
SelectChannelConnector@0.0.0.0:35862
2013-01-23 17:11:38,454 [main] INFO util.InternalHttpServer: Server started
2013-01-23 17:11:38,468 [Heartbeat] INFO agent.MultiMasterRPC: No active
master RPC connection
2013-01-23 17:11:38,495 [Heartbeat] INFO agent.ThriftMasterRPC: Connected
to master at hadoop48:35872
2013-01-23 17:11:38,687 [main] INFO agent.LogicalNodeManager: creating new
logical node Hadoop47
2013-01-23 17:11:38,819 [main] WARN agent.FlumeNode: Flume is using Hadoop
core 1.0.4 which does not support Security / Authentication: null
2013-01-23 17:11:43,529 [Heartbeat] INFO agent.LogicalNodeManager: creating
new logical node agent2
2013-01-23 17:11:43,536 [logicalNode agent2-18] ERROR
connector.DirectDriver: Closing down due to exception on open calls
2013-01-23 17:11:43,537 [logicalNode agent2-18] INFO
connector.DirectDriver: Connector logicalNode agent2-18 exited with error:
Attempting to open a stub sink 'fail'!
2013-01-23 17:11:43,537 [logicalNode agent2-18] ERROR
connector.DirectDriver: Exiting driver logicalNode agent2-18 in error state
TailSource | fail because Attempting to open a stub sink 'fail'!
2013-01-23 17:11:43,540 [TailThread-0] INFO text.TailSource: TailThread has
exited
2013-01-23 17:11:43,553 [Check config] INFO agent.LogicalNode: Node config
successfully set to com.cloudera.flume.conf.FlumeConfigData@76a0f9cb
2013-01-23 17:11:43,554 [logicalNode agent2-20] ERROR
connector.DirectDriver: Closing down due to exception on open calls
2013-01-23 17:11:43,554 [logicalNode agent2-20] INFO
connector.DirectDriver: Connector logicalNode agent2-20 exited with error:
Attempting to open a stub sink 'fail'!
2013-01-23 17:11:43,554 [logicalNode agent2-20] ERROR
connector.DirectDriver: Exiting driver logicalNode agent2-20 in error state
TailSource | fail because Attempting to open a stub sink 'fail'!
2013-01-23 17:11:43,554 [TailThread-1] INFO text.TailSource: TailThread has
exited
http://hadoop48:35871/flumemaster.jsp
Node status logical node physical nodehost name statusversionlast seen
delta (s) last seenHadoop46 Hadoop46Hadoop46IDLEnone 2Wed Jan 23 17:07:05
CST 2013 Hadoop47Hadoop47Hadoop47 IDLEnone1Wed Jan 23 17:07:06 CST 2013
Hadoop48Hadoop48Hadoop48 IDLEWed Jan 23 16:49:27 CST 2013 0Wed Jan 23
17:07:08 CST 2013WX-wuwb.TKOffice.local WX-wuwb.TKOffice.local
WX-wuwb.TKOffice.localERROR Wed Jan 23 16:49:27 CST 20130Wed Jan 23
17:07:07 CST 2013 agent2Hadoop47Hadoop47ERROR Wed Jan 23 16:49:27 CST
20131Wed Jan 23 17:07:06 CST 2013
aggent1Hadoop46 Hadoop46IDLEnone2 Wed Jan 23 17:07:05 CST 2013co1co1Hadoop48
LOSTWed Jan 23 09:56:48 CST 2013 1057Wed Jan 23 16:49:30 CST
2013collect1Hadoop48
Hadoop48ERRORWed Jan 23 16:49:27 CST 2013 0Wed Jan 23 17:07:08 CST
2013zhouhh.TKOffice.local
zhouhh.TKOffice.localzhouhh.TKOffice.local IDLEWed Jan 23 16:49:27 CST
20130Wed Jan 23 17:07:07 CST 2013
Physical/Logical Node mapping physical nodelogical node ag1ag1
WX-wuwb.TKOffice.local WX-wuwb.TKOffice.localhadoop48conn hadoop48connag46
ag46 zhouhh.TKOffice.localzhouhh.TKOffice.localag47 ag47co1co1 Hadoop47
Hadoop47,agent2Hadoop48 Hadoop48,collect1collectorcollector h48co2h48co2
h48coll h48collHadoop46Hadoop46,aggent1
some one can tell me what's matter?
thanks!
Andy
Re: ERROR connector.DirectDriver: Exiting driver logicalNode
agent2-18 in error state TailSource | fail because Attempting to open a stub
sink 'fail'!
Posted by 周梦想 <ab...@gmail.com>.
thank you jeong-shik, I will try this:)
Andy
2013/1/23 Jeong-shik Jang <js...@gruter.com>
> Hi Andy,
>
> How about trying "refreshAll" command to see it helps?
>
> It seems that logicalSource and logicalSink are not working perfectly
> especially in finding and mapping logical node to physical node during
> translation process in master node. I think "map Hadoop48 collect1" should
> be done earlier so that master has mapping information for collector1 to
> Hadoop48(physical node) so that translation can be done properly when
> related configs were created.
>
> for example,
> (config) collect1: null | null
>
> map Hadoop48 collect1
>
> (config) collect1: logicalSource |
> collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);
>
> (config) agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> (config) agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
>
> map Hadoop46 agent1
> map Hadoop47 agent2
>
> and, there seems typo in node name; you mapped "aggent1" to Hadoop46 but
> name of the config you had created was "agent1"; it is why aggent1 on
> Hadoop46 is "IDLE".
>
> - JS
>
> On 1/23/13 6:21 PM, 周梦想 wrote:
>
> hello,
> I add logical config to flume:
> agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> collect1: logicalSource |
> collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);
>
> map Hadoop46 agent1
> map Hadoop47 agent2
> map Hadoop48 collect1
>
> ============================
> I run master on Hadoop48,
> it reports:
> 2013-01-23 16:49:27,153 [main] INFO rolling.RollSink: Created RollSink:
> trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@7a960b08]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:27,158 [main] INFO rolling.RollSink: Created RollSink:
> trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1d29109e]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:27,207 [main] INFO master.ZooKeeperConfigStore: Created
> new config at /flume-cfgs/cfg-0000000104
> 2013-01-23 16:49:27,207 [main-EventThread] INFO
> master.ZooKeeperConfigStore: Config was updated - reloading
> 2013-01-23 16:49:27,255 [main] INFO master.ZooKeeperConfigStore: Created
> new config at /flume-cfgs/cfg-0000000105
> 2013-01-23 16:49:27,255 [main-EventThread] INFO
> master.ZooKeeperConfigStore: Config was updated - reloading
> 2013-01-23 16:49:27,307 [main] WARN logical.LogicalConfigurationManager:
> Logical node 'collect1' has not been assigned to a physical node yet
>
> =================
> run agent1 on Hadoop46, it reports:
>
> 2013-01-23 16:49:45,070 [Heartbeat] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1fd54f77]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,073 [Heartbeat] INFO agent.LogicalNodeManager:
> creating new logical node collect1
> 2013-01-23 16:49:45,076 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Closing down due to exception on open calls
> 2013-01-23 16:49:45,076 [logicalNode collect1-18] INFO
> connector.DirectDriver: Connector logicalNode collect1-18 exited with
> error: Attempting to open a Stub Source!
> 2013-01-23 16:49:45,077 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Error closing logicalNode collect1-18 source:
> Attempting to close a Stub Source!
> 2013-01-23 16:49:45,077 [logicalNode collect1-18] INFO rolling.RollSink:
> closing RollSink
> 'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,081 [logicalNode collect1-18] INFO rolling.RollSink:
> double close
> 'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,081 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Exiting driver logicalNode collect1-18 in error
> state StubSource | Collector because Attempting to open a Stub Source!
> 2013-01-23 16:49:45,091 [Check config] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@5780208f]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
>
> ===============
> and agent2 on Hadoop47 , also logs error.
> [zhouhh@Hadoop47 ~]$ flume node
>
> 2013-01-23 17:11:38,454 [main] INFO mortbay.log: Started
> SelectChannelConnector@0.0.0.0:35862
> 2013-01-23 17:11:38,454 [main] INFO util.InternalHttpServer: Server started
> 2013-01-23 17:11:38,468 [Heartbeat] INFO agent.MultiMasterRPC: No active
> master RPC connection
> 2013-01-23 17:11:38,495 [Heartbeat] INFO agent.ThriftMasterRPC: Connected
> to master at hadoop48:35872
> 2013-01-23 17:11:38,687 [main] INFO agent.LogicalNodeManager: creating new
> logical node Hadoop47
> 2013-01-23 17:11:38,819 [main] WARN agent.FlumeNode: Flume is using Hadoop
> core 1.0.4 which does not support Security / Authentication: null
> 2013-01-23 17:11:43,529 [Heartbeat] INFO agent.LogicalNodeManager:
> creating new logical node agent2
> 2013-01-23 17:11:43,536 [logicalNode agent2-18] ERROR
> connector.DirectDriver: Closing down due to exception on open calls
> 2013-01-23 17:11:43,537 [logicalNode agent2-18] INFO
> connector.DirectDriver: Connector logicalNode agent2-18 exited with error:
> Attempting to open a stub sink 'fail'!
> 2013-01-23 17:11:43,537 [logicalNode agent2-18] ERROR
> connector.DirectDriver: Exiting driver logicalNode agent2-18 in error state
> TailSource | fail because Attempting to open a stub sink 'fail'!
> 2013-01-23 17:11:43,540 [TailThread-0] INFO text.TailSource: TailThread
> has exited
> 2013-01-23 17:11:43,553 [Check config] INFO agent.LogicalNode: Node config
> successfully set to com.cloudera.flume.conf.FlumeConfigData@76a0f9cb
> 2013-01-23 17:11:43,554 [logicalNode agent2-20] ERROR
> connector.DirectDriver: Closing down due to exception on open calls
> 2013-01-23 17:11:43,554 [logicalNode agent2-20] INFO
> connector.DirectDriver: Connector logicalNode agent2-20 exited with error:
> Attempting to open a stub sink 'fail'!
> 2013-01-23 17:11:43,554 [logicalNode agent2-20] ERROR
> connector.DirectDriver: Exiting driver logicalNode agent2-20 in error state
> TailSource | fail because Attempting to open a stub sink 'fail'!
> 2013-01-23 17:11:43,554 [TailThread-1] INFO text.TailSource: TailThread
> has exited
>
> http://hadoop48:35871/flumemaster.jsp
> Node status logical node physical node host name status version last
> seen delta (s) last seen Hadoop46 Hadoop46 Hadoop46 IDLE none 2 Wed Jan
> 23 17:07:05 CST 2013 Hadoop47 Hadoop47 Hadoop47 IDLE none 1 Wed Jan 23
> 17:07:06 CST 2013 Hadoop48 Hadoop48 Hadoop48 IDLE Wed Jan 23 16:49:27
> CST 2013 0 Wed Jan 23 17:07:08 CST 2013 WX-wuwb.TKOffice.local
> WX-wuwb.TKOffice.local WX-wuwb.TKOffice.local ERROR Wed Jan 23 16:49:27
> CST 2013 0 Wed Jan 23 17:07:07 CST 2013 agent2 Hadoop47 Hadoop47 ERROR Wed
> Jan 23 16:49:27 CST 2013 1 Wed Jan 23 17:07:06 CST 2013 aggent1 Hadoop46
> Hadoop46 IDLE none 2 Wed Jan 23 17:07:05 CST 2013 co1 co1 Hadoop48 LOST Wed
> Jan 23 09:56:48 CST 2013 1057 Wed Jan 23 16:49:30 CST 2013 collect1
> Hadoop48 Hadoop48 ERROR Wed Jan 23 16:49:27 CST 2013 0 Wed Jan 23
> 17:07:08 CST 2013 zhouhh.TKOffice.local zhouhh.TKOffice.local
> zhouhh.TKOffice.local IDLE Wed Jan 23 16:49:27 CST 2013 0 Wed Jan 23
> 17:07:07 CST 2013
>
> Physical/Logical Node mapping physical node logical node ag1 ag1
> WX-wuwb.TKOffice.local WX-wuwb.TKOffice.local hadoop48conn hadoop48conn
> ag46 ag46 zhouhh.TKOffice.local zhouhh.TKOffice.local ag47 ag47 co1 co1
> Hadoop47 Hadoop47,agent2 Hadoop48 Hadoop48,collect1 collector collector
> h48co2 h48co2 h48coll h48coll Hadoop46 Hadoop46,aggent1
>
>
> some one can tell me what's matter?
>
> thanks!
> Andy
>
>
>
> --
> Jeong-shik Jang / jsjang@gruter.com
> Gruter, Inc., R&D Team Leaderwww.gruter.com
> Enjoy Connecting
>
>
Re: ERROR connector.DirectDriver: Exiting driver logicalNode agent2-18
in error state TailSource | fail because Attempting to open a stub sink 'fail'!
Posted by Jeong-shik Jang <js...@gruter.com>.
Hi Andy,
How about trying "refreshAll" command to see it helps?
It seems that logicalSource and logicalSink are not working perfectly
especially in finding and mapping logical node to physical node during
translation process in master node. I think "map Hadoop48 collect1"
should be done earlier so that master has mapping information for
collector1 to Hadoop48(physical node) so that translation can be done
properly when related configs were created.
for example,
(config) collect1: null | null
map Hadoop48 collect1
(config) collect1: logicalSource |
collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);
(config) agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
(config) agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
map Hadoop46 agent1
map Hadoop47 agent2
and, there seems typo in node name; you mapped "aggent1" to Hadoop46 but
name of the config you had created was "agent1"; it is why aggent1 on
Hadoop46 is "IDLE".
- JS
On 1/23/13 6:21 PM, ??? wrote:
> hello,
> I add logical config to flume:
> agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> collect1: logicalSource |
> collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);
>
> map Hadoop46 agent1
> map Hadoop47 agent2
> map Hadoop48 collect1
>
> ============================
> I run master on Hadoop48,
> it reports:
> 2013-01-23 16:49:27,153 [main] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@7a960b08]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:27,158 [main] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1d29109e]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:27,207 [main] INFO master.ZooKeeperConfigStore:
> Created new config at /flume-cfgs/cfg-0000000104
> 2013-01-23 16:49:27,207 [main-EventThread] INFO
> master.ZooKeeperConfigStore: Config was updated - reloading
> 2013-01-23 16:49:27,255 [main] INFO master.ZooKeeperConfigStore:
> Created new config at /flume-cfgs/cfg-0000000105
> 2013-01-23 16:49:27,255 [main-EventThread] INFO
> master.ZooKeeperConfigStore: Config was updated - reloading
> 2013-01-23 16:49:27,307 [main] WARN
> logical.LogicalConfigurationManager: Logical node 'collect1' has not
> been assigned to a physical node yet
>
> =================
> run agent1 on Hadoop46, it reports:
>
> 2013-01-23 16:49:45,070 [Heartbeat] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1fd54f77]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,073 [Heartbeat] INFO agent.LogicalNodeManager:
> creating new logical node collect1
> 2013-01-23 16:49:45,076 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Closing down due to exception on open calls
> 2013-01-23 16:49:45,076 [logicalNode collect1-18] INFO
> connector.DirectDriver: Connector logicalNode collect1-18 exited with
> error: Attempting to open a Stub Source!
> 2013-01-23 16:49:45,077 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Error closing logicalNode collect1-18 source:
> Attempting to close a Stub Source!
> 2013-01-23 16:49:45,077 [logicalNode collect1-18] INFO
> rolling.RollSink: closing RollSink
> 'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,081 [logicalNode collect1-18] INFO
> rolling.RollSink: double close
> 'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,081 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Exiting driver logicalNode collect1-18 in
> error state StubSource | Collector because Attempting to open a Stub
> Source!
> 2013-01-23 16:49:45,091 [Check config] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@5780208f]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
>
> ===============
> and agent2 on Hadoop47 , also logs error.
> [zhouhh@Hadoop47 ~]$ flume node
>
> 2013-01-23 17:11:38,454 [main] INFO mortbay.log: Started
> SelectChannelConnector@0.0.0.0:35862
> <http://SelectChannelConnector@0.0.0.0:35862>
> 2013-01-23 17:11:38,454 [main] INFO util.InternalHttpServer: Server
> started
> 2013-01-23 17:11:38,468 [Heartbeat] INFO agent.MultiMasterRPC: No
> active master RPC connection
> 2013-01-23 17:11:38,495 [Heartbeat] INFO agent.ThriftMasterRPC:
> Connected to master at hadoop48:35872
> 2013-01-23 17:11:38,687 [main] INFO agent.LogicalNodeManager: creating
> new logical node Hadoop47
> 2013-01-23 17:11:38,819 [main] WARN agent.FlumeNode: Flume is using
> Hadoop core 1.0.4 which does not support Security / Authentication: null
> 2013-01-23 17:11:43,529 [Heartbeat] INFO agent.LogicalNodeManager:
> creating new logical node agent2
> 2013-01-23 17:11:43,536 [logicalNode agent2-18] ERROR
> connector.DirectDriver: Closing down due to exception on open calls
> 2013-01-23 17:11:43,537 [logicalNode agent2-18] INFO
> connector.DirectDriver: Connector logicalNode agent2-18 exited with
> error: Attempting to open a stub sink 'fail'!
> 2013-01-23 17:11:43,537 [logicalNode agent2-18] ERROR
> connector.DirectDriver: Exiting driver logicalNode agent2-18 in error
> state TailSource | fail because Attempting to open a stub sink 'fail'!
> 2013-01-23 17:11:43,540 [TailThread-0] INFO text.TailSource:
> TailThread has exited
> 2013-01-23 17:11:43,553 [Check config] INFO agent.LogicalNode: Node
> config successfully set to
> com.cloudera.flume.conf.FlumeConfigData@76a0f9cb
> 2013-01-23 17:11:43,554 [logicalNode agent2-20] ERROR
> connector.DirectDriver: Closing down due to exception on open calls
> 2013-01-23 17:11:43,554 [logicalNode agent2-20] INFO
> connector.DirectDriver: Connector logicalNode agent2-20 exited with
> error: Attempting to open a stub sink 'fail'!
> 2013-01-23 17:11:43,554 [logicalNode agent2-20] ERROR
> connector.DirectDriver: Exiting driver logicalNode agent2-20 in error
> state TailSource | fail because Attempting to open a stub sink 'fail'!
> 2013-01-23 17:11:43,554 [TailThread-1] INFO text.TailSource:
> TailThread has exited
>
> http://hadoop48:35871/flumemaster.jsp
>
>
> Node status
>
> logical node physical node host name status version last seen
> delta (s) last seen
> Hadoop46 Hadoop46 Hadoop46 IDLE none 2 Wed Jan 23 17:07:05 CST 2013
> Hadoop47 Hadoop47 Hadoop47 IDLE none 1 Wed Jan 23 17:07:06 CST 2013
> Hadoop48 Hadoop48 Hadoop48 IDLE Wed Jan 23 16:49:27 CST 2013 0
> Wed Jan 23 17:07:08 CST 2013
> WX-wuwb.TKOffice.local WX-wuwb.TKOffice.local
> WX-wuwb.TKOffice.local ERROR Wed Jan 23 16:49:27 CST 2013 0 Wed
> Jan 23 17:07:07 CST 2013
> agent2 Hadoop47 Hadoop47 ERROR Wed Jan 23 16:49:27 CST 2013 1
> Wed Jan 23 17:07:06 CST 2013
> aggent1 Hadoop46 Hadoop46 IDLE none 2 Wed Jan 23 17:07:05 CST 2013
> co1 co1 Hadoop48 LOST Wed Jan 23 09:56:48 CST 2013 1057 Wed Jan
> 23 16:49:30 CST 2013
> collect1 Hadoop48 Hadoop48 ERROR Wed Jan 23 16:49:27 CST 2013 0
> Wed Jan 23 17:07:08 CST 2013
> zhouhh.TKOffice.local zhouhh.TKOffice.local
> zhouhh.TKOffice.local IDLE Wed Jan 23 16:49:27 CST 2013 0 Wed Jan
> 23 17:07:07 CST 2013
>
>
>
> Physical/Logical Node mapping
>
> physical node logical node
> ag1 ag1
> WX-wuwb.TKOffice.local WX-wuwb.TKOffice.local
> hadoop48conn hadoop48conn
> ag46 ag46
> zhouhh.TKOffice.local zhouhh.TKOffice.local
> ag47 ag47
> co1 co1
> Hadoop47 Hadoop47,agent2
> Hadoop48 Hadoop48,collect1
> collector collector
> h48co2 h48co2
> h48coll h48coll
> Hadoop46 Hadoop46,aggent1
>
>
> some one can tell me what's matter?
>
> thanks!
> Andy
--
Jeong-shik Jang / jsjang@gruter.com
Gruter, Inc., R&D Team Leader
www.gruter.com
Enjoy Connecting