How to run multiple flow on single physical node

[纪君祥 <zh...@alipay.com>]

Hi all ,

I failed to do flow isolation as the steps in http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_flow_isolation


I want to run multiple flow one physical node.
Right now, what I think is as following:

On node host1:

#flume node �Cn host1-1
#flume node �Cn host1-2
#flume node �Cn host1-3

When I using map method, I found one bug, please refer to https://issues.apache.org/jira/browse/FLUME-763


Doing as http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_flow_isolation, but failed
Environment:
1 flume master, 3 flume nodes
OS: Redhat Enterprise Linux
Flume version: 0.9.4 release

Steps:

I want to do as follow
exec config hbase2 flow1 text("/home/hadoop/test.txt")       autoE2EChain
exec config hbase2 flow2 text("/home/hadoop/test_character") autoE2EChain
exec config hbase3 flow1 text("/home/hadoop/test.txt")       autoE2EChain
exec config hbase3 flow2 text("/home/hadoop/test_character") autoE2EChain
exec config hbase6 flow1 autoCollectorSource text("/home/hadoop/output_test.txt")
exec config hbase6 flow2 autoCollectorSource text("/home/hadoop/output_test_character")


when I input in flume shell

[flume hbase1:35873:45678] exec config hbase2 flow1 text("/home/hadoop/test.txt") autoE2EChain
Failed to run command 'exec config hbase2 flow1 text("/home/hadoop/test.txt") autoE2EChain' due to null
11/09/14 21:22:38 ERROR util.FlumeShell: Failed to run command 'exec config hbase2 flow1 text("/home/hadoop/test.txt") autoE2EChain'


Thanks
Longda

________________________________

This email (including any attachments) is confidential and may be legally privileged. If you received this email in error, please delete it immediately and do not copy it or use it for any purpose or disclose its contents to any other person. Thank you.

������(�����κθ���)���ܺ��л������ϲ��ܷ��ɱ���������������ȷ���ռ��ˣ���������ɾ�����ʼ����벻Ҫ�������ʽ��и��Ʋ������κ�������;����͸¶���ʼ�֮���ݡ�лл��

Re: How to run multiple flow on single physical node

[Jeff Hansen <ds...@gmail.com>]

Yes, I love those error messages -- 'Failed to run command "xyz" due
to null' is never particularly helpful.

I think you just need to put single quotes around your source -- try:

exec config hbase2 flow2 'text("/home/hadoop/test_character")' autoE2EChain

I found the syntax very frustrating until I rolled up my sleeves and
read through the antlr grammar files (of course I had to skim through
a few chapters of an antlr book to understand what I was looking
at...).  Now things make a lot more sense (or rather I understand why
some things don't make sense...)



2011/9/16 纪君祥 <zh...@alipay.com>:
> Hi all ,
>
>
>
> I failed to do flow isolation as the steps in
> http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_flow_isolation
>
>
>
>
>
> I want to run multiple flow one physical node.
>
> Right now, what I think is as following:
>
>
>
> On node host1:
>
>
>
> #flume node -n host1-1
>
> #flume node -n host1-2
>
> #flume node -n host1-3
>
>
>
> When I using map method, I found one bug, please refer to
> https://issues.apache.org/jira/browse/FLUME-763
>
>
>
>
>
> Doing as
> http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_flow_isolation,
> but failed
>
> Environment:
>
> 1 flume master, 3 flume nodes
>
> OS: Redhat Enterprise Linux
>
> Flume version: 0.9.4 release
>
>
>
> Steps:
>
>
>
> I want to do as follow
>
> exec config hbase2 flow1 text("/home/hadoop/test.txt")       autoE2EChain
>
> exec config hbase2 flow2 text("/home/hadoop/test_character") autoE2EChain
>
> exec config hbase3 flow1 text("/home/hadoop/test.txt")       autoE2EChain
>
> exec config hbase3 flow2 text("/home/hadoop/test_character") autoE2EChain
>
> exec config hbase6 flow1 autoCollectorSource
> text("/home/hadoop/output_test.txt")
>
> exec config hbase6 flow2 autoCollectorSource
> text("/home/hadoop/output_test_character")
>
>
>
>
>
> when I input in flume shell
>
>
>
> [flume hbase1:35873:45678] exec config hbase2 flow1
> text("/home/hadoop/test.txt") autoE2EChain
>
> Failed to run command 'exec config hbase2 flow1
> text("/home/hadoop/test.txt") autoE2EChain' due to null
>
> 11/09/14 21:22:38 ERROR util.FlumeShell: Failed to run command 'exec config
> hbase2 flow1 text("/home/hadoop/test.txt") autoE2EChain'
>
>
>
>
>
> Thanks
>
> Longda
>
> ________________________________
> This email (including any attachments) is confidential and may be legally
> privileged. If you received this email in error, please delete it
> immediately and do not copy it or use it for any purpose or disclose its
> contents to any other person. Thank you.
>
> 本电邮(包括任何附件)可能含有机密资料并受法律保护。如您不是正确的收件人，请您立即删除本邮件。请不要将本电邮进行复制并用作任何其他用途、或透露本邮件之内容。谢谢。
>