You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Binita Bharati <bi...@gmail.com> on 2014/03/17 08:49:05 UTC
Storm - field grouping question for Class instance tuples
Hi ,
I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has
a String field - class1Key1.
I need to segregate all tuples having same value for
a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2.
How do I go about defining such a topology ? I have seen egs where
tuples emitted were simple string values. But, my emitted tuple is a
Class instance , and the field on which I want to do fieldGrouping is
a particular field of that Class.
(I tried defining a equals and hashcode method in a.b.c.MyClass1.
Didn't work as expected..)
Thanks
Binita
Re: Storm - field grouping question for Class instance tuples
Posted by Binita Bharati <bi...@gmail.com>.
Thanks Srinath, I have already started using this approach. ( i.e
emitting multiple tuples , and doing field grouping in respective
bolts for my required string key).
Thought there could be some other way too.
On Wed, Mar 19, 2014 at 9:31 PM, Srinath C <sr...@gmail.com> wrote:
> You can have Bolt1 do:
>
> public void execute(Tuple input) {
> Class1 class1Instance = (Class1)input.getValueByField("JmsSpoutTuple");
> collector.emit(new Values(class1Instance.class1Key1, class1Instance)); //
> emit 2 values - class1Key1 and class1Instance
> }
>
> Then do field grouping for "class1Key1" into Bolt2.
>
>
>
> On Wed, Mar 19, 2014 at 9:26 PM, Binita Bharati <bi...@gmail.com>
> wrote:
>>
>> Anyone has any idea on this please ?
>>
>> Thanks
>>
>> On Mon, Mar 17, 2014 at 1:34 PM, Binita Bharati
>> <bi...@gmail.com> wrote:
>> > Code sample :
>> >
>> > Topology :
>> > ===================================
>> > TopologyBuilder builder = new TopologyBuilder();
>> >
>> > builder.setSpout("epd-spout",new MyJmsSpout());
>> >
>> > builder.setBolt("epd-CPY001-bolt1", new Bolt1("CPY001"),
>> > parallelismHintMap.get("CPY001"))
>> > .shuffleGrouping("epd-spout");
>> >
>> > builder.setBolt("epd-CPY001-bolt2", new Bolt2("CPY001"),
>> > parallelismHintMap.get("CPY001"))
>> > .fieldsGrouping("epd-CPY001-bolt1", new Fields("Bolt1Tuple"));
>> >
>> >
>> > Bolt1:
>> > ====================================
>> > public void declareOutputFields(OutputFieldsDeclarer declarer) {
>> > // TODO Auto-generated method stub
>> > declarer.declare(new Fields("Bolt1Tuple"));
>> > }
>> >
>> > public void execute(Tuple input) {
>> > Class1 class1Instance = (Class1)input.getValueByField("JmsSpoutTuple");
>> > collector.emit(new Values(class1Instance));
>> > }
>> >
>> >
>> > Bolt2:
>> > ======================================
>> > public void execute(Tuple input) {
>> > Class1 class1Instance = (Class1)input.getValueByField("Bolt1Tuple");
>> >
>> > }
>> >
>> > On Mon, Mar 17, 2014 at 1:19 PM, Binita Bharati
>> > <bi...@gmail.com> wrote:
>> >> Hi ,
>> >>
>> >> I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has
>> >> a String field - class1Key1.
>> >> I need to segregate all tuples having same value for
>> >> a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2.
>> >>
>> >> How do I go about defining such a topology ? I have seen egs where
>> >> tuples emitted were simple string values. But, my emitted tuple is a
>> >> Class instance , and the field on which I want to do fieldGrouping is
>> >> a particular field of that Class.
>> >>
>> >> (I tried defining a equals and hashcode method in a.b.c.MyClass1.
>> >> Didn't work as expected..)
>> >>
>> >> Thanks
>> >> Binita
>
>
Re: Storm - field grouping question for Class instance tuples
Posted by Srinath C <sr...@gmail.com>.
You can have Bolt1 do:
public void execute(Tuple input) {
Class1 class1Instance = (Class1)input.getValueByField("JmsSpoutTuple");
collector.emit(new Values(class1Instance.class1Key1, class1Instance)); //
emit 2 values - class1Key1 and class1Instance
}
Then do field grouping for "class1Key1" into Bolt2.
On Wed, Mar 19, 2014 at 9:26 PM, Binita Bharati <bi...@gmail.com>wrote:
> Anyone has any idea on this please ?
>
> Thanks
>
> On Mon, Mar 17, 2014 at 1:34 PM, Binita Bharati
> <bi...@gmail.com> wrote:
> > Code sample :
> >
> > Topology :
> > ===================================
> > TopologyBuilder builder = new TopologyBuilder();
> >
> > builder.setSpout("epd-spout",new MyJmsSpout());
> >
> > builder.setBolt("epd-CPY001-bolt1", new Bolt1("CPY001"),
> > parallelismHintMap.get("CPY001"))
> > .shuffleGrouping("epd-spout");
> >
> > builder.setBolt("epd-CPY001-bolt2", new Bolt2("CPY001"),
> > parallelismHintMap.get("CPY001"))
> > .fieldsGrouping("epd-CPY001-bolt1", new Fields("Bolt1Tuple"));
> >
> >
> > Bolt1:
> > ====================================
> > public void declareOutputFields(OutputFieldsDeclarer declarer) {
> > // TODO Auto-generated method stub
> > declarer.declare(new Fields("Bolt1Tuple"));
> > }
> >
> > public void execute(Tuple input) {
> > Class1 class1Instance = (Class1)input.getValueByField("JmsSpoutTuple");
> > collector.emit(new Values(class1Instance));
> > }
> >
> >
> > Bolt2:
> > ======================================
> > public void execute(Tuple input) {
> > Class1 class1Instance = (Class1)input.getValueByField("Bolt1Tuple");
> >
> > }
> >
> > On Mon, Mar 17, 2014 at 1:19 PM, Binita Bharati
> > <bi...@gmail.com> wrote:
> >> Hi ,
> >>
> >> I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has
> >> a String field - class1Key1.
> >> I need to segregate all tuples having same value for
> >> a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2.
> >>
> >> How do I go about defining such a topology ? I have seen egs where
> >> tuples emitted were simple string values. But, my emitted tuple is a
> >> Class instance , and the field on which I want to do fieldGrouping is
> >> a particular field of that Class.
> >>
> >> (I tried defining a equals and hashcode method in a.b.c.MyClass1.
> >> Didn't work as expected..)
> >>
> >> Thanks
> >> Binita
>
Re: Storm - field grouping question for Class instance tuples
Posted by Binita Bharati <bi...@gmail.com>.
Anyone has any idea on this please ?
Thanks
On Mon, Mar 17, 2014 at 1:34 PM, Binita Bharati
<bi...@gmail.com> wrote:
> Code sample :
>
> Topology :
> ===================================
> TopologyBuilder builder = new TopologyBuilder();
>
> builder.setSpout("epd-spout",new MyJmsSpout());
>
> builder.setBolt("epd-CPY001-bolt1", new Bolt1("CPY001"),
> parallelismHintMap.get("CPY001"))
> .shuffleGrouping("epd-spout");
>
> builder.setBolt("epd-CPY001-bolt2", new Bolt2("CPY001"),
> parallelismHintMap.get("CPY001"))
> .fieldsGrouping("epd-CPY001-bolt1", new Fields("Bolt1Tuple"));
>
>
> Bolt1:
> ====================================
> public void declareOutputFields(OutputFieldsDeclarer declarer) {
> // TODO Auto-generated method stub
> declarer.declare(new Fields("Bolt1Tuple"));
> }
>
> public void execute(Tuple input) {
> Class1 class1Instance = (Class1)input.getValueByField("JmsSpoutTuple");
> collector.emit(new Values(class1Instance));
> }
>
>
> Bolt2:
> ======================================
> public void execute(Tuple input) {
> Class1 class1Instance = (Class1)input.getValueByField("Bolt1Tuple");
>
> }
>
> On Mon, Mar 17, 2014 at 1:19 PM, Binita Bharati
> <bi...@gmail.com> wrote:
>> Hi ,
>>
>> I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has
>> a String field - class1Key1.
>> I need to segregate all tuples having same value for
>> a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2.
>>
>> How do I go about defining such a topology ? I have seen egs where
>> tuples emitted were simple string values. But, my emitted tuple is a
>> Class instance , and the field on which I want to do fieldGrouping is
>> a particular field of that Class.
>>
>> (I tried defining a equals and hashcode method in a.b.c.MyClass1.
>> Didn't work as expected..)
>>
>> Thanks
>> Binita
Re: Storm - field grouping question for Class instance tuples
Posted by Binita Bharati <bi...@gmail.com>.
Code sample :
Topology :
===================================
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("epd-spout",new MyJmsSpout());
builder.setBolt("epd-CPY001-bolt1", new Bolt1("CPY001"),
parallelismHintMap.get("CPY001"))
.shuffleGrouping("epd-spout");
builder.setBolt("epd-CPY001-bolt2", new Bolt2("CPY001"),
parallelismHintMap.get("CPY001"))
.fieldsGrouping("epd-CPY001-bolt1", new Fields("Bolt1Tuple"));
Bolt1:
====================================
public void declareOutputFields(OutputFieldsDeclarer declarer) {
// TODO Auto-generated method stub
declarer.declare(new Fields("Bolt1Tuple"));
}
public void execute(Tuple input) {
Class1 class1Instance = (Class1)input.getValueByField("JmsSpoutTuple");
collector.emit(new Values(class1Instance));
}
Bolt2:
======================================
public void execute(Tuple input) {
Class1 class1Instance = (Class1)input.getValueByField("Bolt1Tuple");
}
On Mon, Mar 17, 2014 at 1:19 PM, Binita Bharati
<bi...@gmail.com> wrote:
> Hi ,
>
> I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has
> a String field - class1Key1.
> I need to segregate all tuples having same value for
> a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2.
>
> How do I go about defining such a topology ? I have seen egs where
> tuples emitted were simple string values. But, my emitted tuple is a
> Class instance , and the field on which I want to do fieldGrouping is
> a particular field of that Class.
>
> (I tried defining a equals and hashcode method in a.b.c.MyClass1.
> Didn't work as expected..)
>
> Thanks
> Binita