conditional streaming

[clay teahouse <cl...@gmail.com>]

Hello All,

My data source can have multiple formats, except that all records share the
first field. Based on the value of this field, I want to generate a
separate stream that goes to a particular bolt for special processing. Now
my question is how one does conditional streaming, based on a particular
field in the stream?

For example:

Bolt A if field F="X" generate stream1 with certain fields that goes to
Bolt B
Bolt A  if field F="Y" generate stream2 with certain fields that goes to
Bolt C

I am sorry if this is a trivial question, but I don't seem to be able to do
this in a straightforward way and I can't find an example that helps.

thanks,
Clay

Re: conditional streaming

[Susheel Kumar Gadalay <sk...@gmail.com>]

You can do like this in your bolt for methods declareOutputFields, execute.

  public void declareOutputFields(OutputFieldsDeclarer declarer) {
    declarer.declareStream("signals", new Fields("action"));
    //declarer.declare(new Fields("word"));
  }

  public void execute(Tuple input) {
    String str = input.getString(0);

    if(str.equals("EOF")){
      collector.emit("signals",new Values("dumpValues"));
      //List a = new ArrayList();
      //a.add(input);
      //collector.emit(a,new Values(str));
    }
    // Acknowledge the tuple
    collector.ack(input);
  }

Then in the downstream bolt

  @Override
  public void execute(Tuple input) {

      if (input.getSourceStreamId().equals("signals")) {
      ......
     }

    //Set the tuple as Acknowledge
    collector.ack(input);
  }

On 10/30/14, clay teahouse <cl...@gmail.com> wrote:
> Hello All,
>
> My data source can have multiple formats, except that all records share the
> first field. Based on the value of this field, I want to generate a
> separate stream that goes to a particular bolt for special processing. Now
> my question is how one does conditional streaming, based on a particular
> field in the stream?
>
> For example:
>
> Bolt A if field F="X" generate stream1 with certain fields that goes to
> Bolt B
> Bolt A  if field F="Y" generate stream2 with certain fields that goes to
> Bolt C
>
> I am sorry if this is a trivial question, but I don't seem to be able to do
> this in a straightforward way and I can't find an example that helps.
>
> thanks,
> Clay
>

Re: conditional streaming

[Kobi Salant <ko...@liveperson.com>]

Hi Clay,

you can use streams, when building your topology connect bolt A to bolt B
by stream S1 and similarly A to C by S2. It is an additional parameter to
the shuffle function.

When emitting tuple from bolt A add the relevant stream to the emit method.

Kobi

On Thu, Oct 30, 2014 at 1:13 PM, clay teahouse <cl...@gmail.com>
wrote:

> Hello All,
>
> My data source can have multiple formats, except that all records share
> the first field. Based on the value of this field, I want to generate a
> separate stream that goes to a particular bolt for special processing. Now
> my question is how one does conditional streaming, based on a particular
> field in the stream?
>
> For example:
>
> Bolt A if field F="X" generate stream1 with certain fields that goes to
> Bolt B
> Bolt A  if field F="Y" generate stream2 with certain fields that goes to
> Bolt C
>
> I am sorry if this is a trivial question, but I don't seem to be able to
> do this in a straightforward way and I can't find an example that helps.
>
> thanks,
> Clay
>
>

-- 
This message may contain confidential and/or privileged information. 
If you are not the addressee or authorized to receive this on behalf of the 
addressee you must not use, copy, disclose or take action based on this 
message or any information herein. 
If you have received this message in error, please advise the sender 
immediately by reply email and delete this message. Thank you.