How to use Flume as an ad hoc data collector in synchronized way?

[Li Yunpeng <lo...@gmail.com>]

Hi there,
I'm new to this tool, it's really cool. And I was wandering if we could
use Flume as a ad hoc data collector or log search engine, what do I
mean is that, each time we issue a new configuration, can we collect its
result like calling a method and waiting for return vale in synchronized
way, it seems that we have to get the result in an asynchronized way,
such as listening to a port(which will induce a protocol to encode the
request ids).
I knew it is designed for consolidate logs, however, what if I do not
want to collect the data, say we build a web portal(can be a web
service) to grep data on distributed environment, and then we issue a
flume configuration in programmatic way, and display on the results
through a web portal.
e.g.:
List<OutputThrift> outputs = flumeMasterNodeThrift.synchSink(new
ExecSorce("cat /usr/services | grep 'tcp'"),decorators);
for(OutputThrift output:outputs){
//...
}
Any suggestion how to do it?

Thanks a lot

YP Li