You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Kashyap Mhaisekar <ka...@gmail.com> on 2015/03/24 16:58:54 UTC
Storm DRPC Performance
Hi,
Am doing a feasibility as to using the storm DRPC to power our backend APIs
and have run into some problems. The core functionality that needs to be
executed takes around 2 ms to execute but in storm takes close to 400 msec.
Requesting an approach for them -
1. The LinearDRPCTopologyBuilder uses DRPCSpout by default. Is there a way
to increase the no. of instances on them? I see that spout always has
executors at 1. The problem is that when more than one request comes in at
the same time, the times go up drastically.
2. The bolts are defined as bolt0, bolt1, bolt2 etc. I used only 3 bolts
but there are 5 bolts shown (bolt0 to bolt4). Is there a way I can change
the names of the bolts shown in storm UI to more meaningful way the way it
happens in regular storm?
3. For some of the bolts, the Process Latency is very very high compared to
execute latency. I am trying to figure out the reason but am unsuccessful.
4. The Complete Latency on the spout is very high at 500+ msc. I am trying
to figure out the reason but am unsuccessful.
Configurations used as below: :
config.put(Config.DRPC_WORKER_THREADS,64);
config.setNumWorkers(50);
config.setMaxTaskParallelism(50);
config.setMaxSpoutPending(5000);
config.put(Config.TOPOLOGY_ACKER_EXECUTORS, 50);
config.put(Config.TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE, 16384);
config.put(Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE, 16384);
Supervisor slot ports: [6700 6701 6702 6703 6704 6705 6706 6707 6708 6709
6710 6711]
I have cluster of 5 servers with 8 GB and 4 cores. Of the 5, 3 servers have
DRPC running.
*Storm UI Details:*
[image: Inline image 1]
Please do help me out.
Regards,
Kashyap
Re: Storm DRPC Performance
Posted by Kashyap Mhaisekar <ka...@gmail.com>.
Guys Help!!!
On Tue, Mar 24, 2015 at 1:18 PM, Kashyap Mhaisekar <ka...@gmail.com>
wrote:
> Jia,
> Because the BatchBolt is not supported by TopologyBuilder and we need to
> use LinearDRPCTopologyBuilder. I believe this was given as a comment by
> Nathan Marz in some mail...
> Thanks
> Kashyap
>
> On Tue, Mar 24, 2015 at 12:16 PM, <xu...@gmail.com> wrote:
>
>> I am no expert on this, but for your first question, if you want to have
>> more spouts for the drpc, why not use a regular TopologyBuilder?
>>
>> DRPCSpout drpcSpout;
>> if (mode.equals("LocalMode"))
>> {
>> LocalDRPC drpc = new LocalDRPC();
>> drpcSpout = new DRPCSpout(“name", drpc);
>> } else
>> {
>> drpcSpout = new DRPCSpout(“name");
>> topoConf.put(Config.DRPC_SERVERS, Arrays.asList(“drpcServer"));
>> }
>>
>> TopologyBuilder builder = new TopologyBuilder();
>>
>> builder.setSpout("drpc", drpcSpout, 10);
>>
>>
>> -Jia
>>
>>
>> On Tue, Mar 24, 2015 at 9:45 AM, Kashyap Mhaisekar <ka...@gmail.com>
>> wrote:
>>
>>> Brian,
>>> Thanks for the suggestion. The 2 ms operation will need to be iterated
>>> over a million times :) and thats where we wanted to parallelize things and
>>> hence evaluating storm.
>>>
>>> Will look into this option too.
>>>
>>> Regards,
>>> Kashyap
>>>
>>> On Tue, Mar 24, 2015 at 11:06 AM, Brian O'Neill <bo...@alumni.brown.edu>
>>> wrote:
>>>
>>>>
>>>> Just a perspective on this…
>>>>
>>>> We looked at DRPC, and decided *NOT* to go that route. For small
>>>> computations (< 2ms), the network hop itself can add an order of magnitude
>>>> to response times. (also much simpler to keep the computation within a
>>>> single JVM — if you can)
>>>>
>>>> Instead, we are looking to embed Storm directly within our web services.
>>>> So far, we’ve embedded Trident State. We may look at topologies next.
>>>>
>>>> More thoughts on the topic:
>>>>
>>>> http://brianoneill.blogspot.com/2015/03/delta-architectures-unifying-lambda.html
>>>>
>>>> (and I apologize for NOT answering your question =)
>>>>
>>>> -brian
>>>>
>>>> ---
>>>>
>>>> *Brian O'Neill *
>>>>
>>>> Chief Technology Officer
>>>>
>>>> Health Market Science, a LexisNexis Company
>>>>
>>>> 215.588.6024 Mobile • @boneill42 <http://www.twitter.com/boneill42>
>>>>
>>>>
>>>> This information transmitted in this email message is for the
>>>> intended recipient only and may contain confidential and/or privileged
>>>> material. If you received this email in error and are not the intended
>>>> recipient, or the person responsible to deliver it to the intended
>>>> recipient, please contact the sender at the email above and delete this
>>>> email and any attachments and destroy any copies thereof. Any review,
>>>> retransmission, dissemination, copying or other use of, or taking any
>>>> action in reliance upon, this information by persons or entities other than
>>>> the intended recipient is strictly prohibited.
>>>>
>>>>
>>>>
>>>>
>>>> From: Kashyap Mhaisekar <ka...@gmail.com>
>>>> Reply-To: <us...@storm.apache.org>
>>>> Date: Tuesday, March 24, 2015 at 11:58 AM
>>>> To: "user@storm.incubator.apache.org" <us...@storm.incubator.apache.org>,
>>>> <us...@storm.apache.org>
>>>> Subject: Storm DRPC Performance
>>>>
>>>> Hi,
>>>> Am doing a feasibility as to using the storm DRPC to power our backend
>>>> APIs and have run into some problems. The core functionality that needs to
>>>> be executed takes around 2 ms to execute but in storm takes close to 400
>>>> msec. Requesting an approach for them -
>>>>
>>>> 1. The LinearDRPCTopologyBuilder uses DRPCSpout by default. Is there a
>>>> way to increase the no. of instances on them? I see that spout always has
>>>> executors at 1. The problem is that when more than one request comes in at
>>>> the same time, the times go up drastically.
>>>> 2. The bolts are defined as bolt0, bolt1, bolt2 etc. I used only 3
>>>> bolts but there are 5 bolts shown (bolt0 to bolt4). Is there a way I can
>>>> change the names of the bolts shown in storm UI to more meaningful way the
>>>> way it happens in regular storm?
>>>> 3. For some of the bolts, the Process Latency is very very high
>>>> compared to execute latency. I am trying to figure out the reason but am
>>>> unsuccessful.
>>>> 4. The Complete Latency on the spout is very high at 500+ msc. I am
>>>> trying to figure out the reason but am unsuccessful.
>>>>
>>>> Configurations used as below: :
>>>>
>>>> config.put(Config.DRPC_WORKER_THREADS,64);
>>>> config.setNumWorkers(50);
>>>> config.setMaxTaskParallelism(50);
>>>> config.setMaxSpoutPending(5000);
>>>> config.put(Config.TOPOLOGY_ACKER_EXECUTORS, 50);
>>>> config.put(Config.TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE, 16384);
>>>> config.put(Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE, 16384);
>>>> Supervisor slot ports: [6700 6701 6702 6703 6704 6705 6706 6707 6708
>>>> 6709 6710 6711]
>>>> I have cluster of 5 servers with 8 GB and 4 cores. Of the 5, 3 servers
>>>> have DRPC running.
>>>>
>>>>
>>>> *Storm UI Details:*
>>>>
>>>> <image.png>
>>>>
>>>>
>>>> Please do help me out.
>>>>
>>>> Regards,
>>>> Kashyap
>>>>
>>>
>>>
>>
>
Re: Storm DRPC Performance
Posted by Kashyap Mhaisekar <ka...@gmail.com>.
Jia,
Because the BatchBolt is not supported by TopologyBuilder and we need to
use LinearDRPCTopologyBuilder. I believe this was given as a comment by
Nathan Marz in some mail...
Thanks
Kashyap
On Tue, Mar 24, 2015 at 12:16 PM, <xu...@gmail.com> wrote:
> I am no expert on this, but for your first question, if you want to have
> more spouts for the drpc, why not use a regular TopologyBuilder?
>
> DRPCSpout drpcSpout;
> if (mode.equals("LocalMode"))
> {
> LocalDRPC drpc = new LocalDRPC();
> drpcSpout = new DRPCSpout(“name", drpc);
> } else
> {
> drpcSpout = new DRPCSpout(“name");
> topoConf.put(Config.DRPC_SERVERS, Arrays.asList(“drpcServer"));
> }
>
> TopologyBuilder builder = new TopologyBuilder();
>
> builder.setSpout("drpc", drpcSpout, 10);
>
>
> -Jia
>
>
> On Tue, Mar 24, 2015 at 9:45 AM, Kashyap Mhaisekar <ka...@gmail.com>
> wrote:
>
>> Brian,
>> Thanks for the suggestion. The 2 ms operation will need to be iterated
>> over a million times :) and thats where we wanted to parallelize things and
>> hence evaluating storm.
>>
>> Will look into this option too.
>>
>> Regards,
>> Kashyap
>>
>> On Tue, Mar 24, 2015 at 11:06 AM, Brian O'Neill <bo...@alumni.brown.edu>
>> wrote:
>>
>>>
>>> Just a perspective on this…
>>>
>>> We looked at DRPC, and decided *NOT* to go that route. For small
>>> computations (< 2ms), the network hop itself can add an order of magnitude
>>> to response times. (also much simpler to keep the computation within a
>>> single JVM — if you can)
>>>
>>> Instead, we are looking to embed Storm directly within our web services.
>>> So far, we’ve embedded Trident State. We may look at topologies next.
>>>
>>> More thoughts on the topic:
>>>
>>> http://brianoneill.blogspot.com/2015/03/delta-architectures-unifying-lambda.html
>>>
>>> (and I apologize for NOT answering your question =)
>>>
>>> -brian
>>>
>>> ---
>>>
>>> *Brian O'Neill *
>>>
>>> Chief Technology Officer
>>>
>>> Health Market Science, a LexisNexis Company
>>>
>>> 215.588.6024 Mobile • @boneill42 <http://www.twitter.com/boneill42>
>>>
>>>
>>> This information transmitted in this email message is for the intended
>>> recipient only and may contain confidential and/or privileged material. If
>>> you received this email in error and are not the intended recipient, or the
>>> person responsible to deliver it to the intended recipient, please contact
>>> the sender at the email above and delete this email and any attachments and
>>> destroy any copies thereof. Any review, retransmission, dissemination,
>>> copying or other use of, or taking any action in reliance upon, this
>>> information by persons or entities other than the intended recipient is
>>> strictly prohibited.
>>>
>>>
>>>
>>>
>>> From: Kashyap Mhaisekar <ka...@gmail.com>
>>> Reply-To: <us...@storm.apache.org>
>>> Date: Tuesday, March 24, 2015 at 11:58 AM
>>> To: "user@storm.incubator.apache.org" <us...@storm.incubator.apache.org>,
>>> <us...@storm.apache.org>
>>> Subject: Storm DRPC Performance
>>>
>>> Hi,
>>> Am doing a feasibility as to using the storm DRPC to power our backend
>>> APIs and have run into some problems. The core functionality that needs to
>>> be executed takes around 2 ms to execute but in storm takes close to 400
>>> msec. Requesting an approach for them -
>>>
>>> 1. The LinearDRPCTopologyBuilder uses DRPCSpout by default. Is there a
>>> way to increase the no. of instances on them? I see that spout always has
>>> executors at 1. The problem is that when more than one request comes in at
>>> the same time, the times go up drastically.
>>> 2. The bolts are defined as bolt0, bolt1, bolt2 etc. I used only 3 bolts
>>> but there are 5 bolts shown (bolt0 to bolt4). Is there a way I can change
>>> the names of the bolts shown in storm UI to more meaningful way the way it
>>> happens in regular storm?
>>> 3. For some of the bolts, the Process Latency is very very high compared
>>> to execute latency. I am trying to figure out the reason but am
>>> unsuccessful.
>>> 4. The Complete Latency on the spout is very high at 500+ msc. I am
>>> trying to figure out the reason but am unsuccessful.
>>>
>>> Configurations used as below: :
>>>
>>> config.put(Config.DRPC_WORKER_THREADS,64);
>>> config.setNumWorkers(50);
>>> config.setMaxTaskParallelism(50);
>>> config.setMaxSpoutPending(5000);
>>> config.put(Config.TOPOLOGY_ACKER_EXECUTORS, 50);
>>> config.put(Config.TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE, 16384);
>>> config.put(Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE, 16384);
>>> Supervisor slot ports: [6700 6701 6702 6703 6704 6705 6706 6707 6708
>>> 6709 6710 6711]
>>> I have cluster of 5 servers with 8 GB and 4 cores. Of the 5, 3 servers
>>> have DRPC running.
>>>
>>>
>>> *Storm UI Details:*
>>>
>>> <image.png>
>>>
>>>
>>> Please do help me out.
>>>
>>> Regards,
>>> Kashyap
>>>
>>
>>
>
Re: Storm DRPC Performance
Posted by xu...@gmail.com.
I am no expert on this, but for your first question, if you want to have more spouts for the drpc, why not use a regular TopologyBuilder?
DRPCSpout drpcSpout;
if (mode.equals("LocalMode"))
{
LocalDRPC drpc = new LocalDRPC();
drpcSpout = new DRPCSpout(“name", drpc);
} else
{
drpcSpout = new DRPCSpout(“name");
topoConf.put(Config.DRPC_SERVERS, Arrays.asList(“drpcServer"));
}
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("drpc", drpcSpout, 10);
-Jia
On Tue, Mar 24, 2015 at 9:45 AM, Kashyap Mhaisekar <ka...@gmail.com>
wrote:
> Brian,
> Thanks for the suggestion. The 2 ms operation will need to be iterated
> over a million times :) and thats where we wanted to parallelize things and
> hence evaluating storm.
> Will look into this option too.
> Regards,
> Kashyap
> On Tue, Mar 24, 2015 at 11:06 AM, Brian O'Neill <bo...@alumni.brown.edu>
> wrote:
>>
>> Just a perspective on this…
>>
>> We looked at DRPC, and decided *NOT* to go that route. For small
>> computations (< 2ms), the network hop itself can add an order of magnitude
>> to response times. (also much simpler to keep the computation within a
>> single JVM — if you can)
>>
>> Instead, we are looking to embed Storm directly within our web services.
>> So far, we’ve embedded Trident State. We may look at topologies next.
>>
>> More thoughts on the topic:
>>
>> http://brianoneill.blogspot.com/2015/03/delta-architectures-unifying-lambda.html
>>
>> (and I apologize for NOT answering your question =)
>>
>> -brian
>>
>> ---
>>
>> *Brian O'Neill *
>>
>> Chief Technology Officer
>>
>> Health Market Science, a LexisNexis Company
>>
>> 215.588.6024 Mobile • @boneill42 <http://www.twitter.com/boneill42>
>>
>>
>> This information transmitted in this email message is for the intended
>> recipient only and may contain confidential and/or privileged material. If
>> you received this email in error and are not the intended recipient, or the
>> person responsible to deliver it to the intended recipient, please contact
>> the sender at the email above and delete this email and any attachments and
>> destroy any copies thereof. Any review, retransmission, dissemination,
>> copying or other use of, or taking any action in reliance upon, this
>> information by persons or entities other than the intended recipient is
>> strictly prohibited.
>>
>>
>>
>>
>> From: Kashyap Mhaisekar <ka...@gmail.com>
>> Reply-To: <us...@storm.apache.org>
>> Date: Tuesday, March 24, 2015 at 11:58 AM
>> To: "user@storm.incubator.apache.org" <us...@storm.incubator.apache.org>, <
>> user@storm.apache.org>
>> Subject: Storm DRPC Performance
>>
>> Hi,
>> Am doing a feasibility as to using the storm DRPC to power our backend
>> APIs and have run into some problems. The core functionality that needs to
>> be executed takes around 2 ms to execute but in storm takes close to 400
>> msec. Requesting an approach for them -
>>
>> 1. The LinearDRPCTopologyBuilder uses DRPCSpout by default. Is there a way
>> to increase the no. of instances on them? I see that spout always has
>> executors at 1. The problem is that when more than one request comes in at
>> the same time, the times go up drastically.
>> 2. The bolts are defined as bolt0, bolt1, bolt2 etc. I used only 3 bolts
>> but there are 5 bolts shown (bolt0 to bolt4). Is there a way I can change
>> the names of the bolts shown in storm UI to more meaningful way the way it
>> happens in regular storm?
>> 3. For some of the bolts, the Process Latency is very very high compared
>> to execute latency. I am trying to figure out the reason but am
>> unsuccessful.
>> 4. The Complete Latency on the spout is very high at 500+ msc. I am
>> trying to figure out the reason but am unsuccessful.
>>
>> Configurations used as below: :
>>
>> config.put(Config.DRPC_WORKER_THREADS,64);
>> config.setNumWorkers(50);
>> config.setMaxTaskParallelism(50);
>> config.setMaxSpoutPending(5000);
>> config.put(Config.TOPOLOGY_ACKER_EXECUTORS, 50);
>> config.put(Config.TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE, 16384);
>> config.put(Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE, 16384);
>> Supervisor slot ports: [6700 6701 6702 6703 6704 6705 6706 6707 6708 6709
>> 6710 6711]
>> I have cluster of 5 servers with 8 GB and 4 cores. Of the 5, 3 servers
>> have DRPC running.
>>
>>
>> *Storm UI Details:*
>>
>> [image: Inline image 1]
>>
>> Please do help me out.
>>
>> Regards,
>> Kashyap
>>
Re: Storm DRPC Performance
Posted by Kashyap Mhaisekar <ka...@gmail.com>.
Brian,
Thanks for the suggestion. The 2 ms operation will need to be iterated
over a million times :) and thats where we wanted to parallelize things and
hence evaluating storm.
Will look into this option too.
Regards,
Kashyap
On Tue, Mar 24, 2015 at 11:06 AM, Brian O'Neill <bo...@alumni.brown.edu>
wrote:
>
> Just a perspective on this…
>
> We looked at DRPC, and decided *NOT* to go that route. For small
> computations (< 2ms), the network hop itself can add an order of magnitude
> to response times. (also much simpler to keep the computation within a
> single JVM — if you can)
>
> Instead, we are looking to embed Storm directly within our web services.
> So far, we’ve embedded Trident State. We may look at topologies next.
>
> More thoughts on the topic:
>
> http://brianoneill.blogspot.com/2015/03/delta-architectures-unifying-lambda.html
>
> (and I apologize for NOT answering your question =)
>
> -brian
>
> ---
>
> *Brian O'Neill *
>
> Chief Technology Officer
>
> Health Market Science, a LexisNexis Company
>
> 215.588.6024 Mobile • @boneill42 <http://www.twitter.com/boneill42>
>
>
> This information transmitted in this email message is for the intended
> recipient only and may contain confidential and/or privileged material. If
> you received this email in error and are not the intended recipient, or the
> person responsible to deliver it to the intended recipient, please contact
> the sender at the email above and delete this email and any attachments and
> destroy any copies thereof. Any review, retransmission, dissemination,
> copying or other use of, or taking any action in reliance upon, this
> information by persons or entities other than the intended recipient is
> strictly prohibited.
>
>
>
>
> From: Kashyap Mhaisekar <ka...@gmail.com>
> Reply-To: <us...@storm.apache.org>
> Date: Tuesday, March 24, 2015 at 11:58 AM
> To: "user@storm.incubator.apache.org" <us...@storm.incubator.apache.org>, <
> user@storm.apache.org>
> Subject: Storm DRPC Performance
>
> Hi,
> Am doing a feasibility as to using the storm DRPC to power our backend
> APIs and have run into some problems. The core functionality that needs to
> be executed takes around 2 ms to execute but in storm takes close to 400
> msec. Requesting an approach for them -
>
> 1. The LinearDRPCTopologyBuilder uses DRPCSpout by default. Is there a way
> to increase the no. of instances on them? I see that spout always has
> executors at 1. The problem is that when more than one request comes in at
> the same time, the times go up drastically.
> 2. The bolts are defined as bolt0, bolt1, bolt2 etc. I used only 3 bolts
> but there are 5 bolts shown (bolt0 to bolt4). Is there a way I can change
> the names of the bolts shown in storm UI to more meaningful way the way it
> happens in regular storm?
> 3. For some of the bolts, the Process Latency is very very high compared
> to execute latency. I am trying to figure out the reason but am
> unsuccessful.
> 4. The Complete Latency on the spout is very high at 500+ msc. I am
> trying to figure out the reason but am unsuccessful.
>
> Configurations used as below: :
>
> config.put(Config.DRPC_WORKER_THREADS,64);
> config.setNumWorkers(50);
> config.setMaxTaskParallelism(50);
> config.setMaxSpoutPending(5000);
> config.put(Config.TOPOLOGY_ACKER_EXECUTORS, 50);
> config.put(Config.TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE, 16384);
> config.put(Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE, 16384);
> Supervisor slot ports: [6700 6701 6702 6703 6704 6705 6706 6707 6708 6709
> 6710 6711]
> I have cluster of 5 servers with 8 GB and 4 cores. Of the 5, 3 servers
> have DRPC running.
>
>
> *Storm UI Details:*
>
> [image: Inline image 1]
>
> Please do help me out.
>
> Regards,
> Kashyap
>
Re: Storm DRPC Performance
Posted by Brian O'Neill <bo...@alumni.brown.edu>.
Just a perspective on this
We looked at DRPC, and decided *NOT* to go that route. For small
computations (< 2ms), the network hop itself can add an order of magnitude
to response times. (also much simpler to keep the computation within a
single JVM if you can)
Instead, we are looking to embed Storm directly within our web services.
So far, we¹ve embedded Trident State. We may look at topologies next.
More thoughts on the topic:
http://brianoneill.blogspot.com/2015/03/delta-architectures-unifying-lambda.
html
(and I apologize for NOT answering your question =)
-brian
---
Brian O'Neill
Chief Technology Officer
Health Market Science, a LexisNexis Company
215.588.6024 Mobile @boneill42 <http://www.twitter.com/boneill42>
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
From: Kashyap Mhaisekar <ka...@gmail.com>
Reply-To: <us...@storm.apache.org>
Date: Tuesday, March 24, 2015 at 11:58 AM
To: "user@storm.incubator.apache.org" <us...@storm.incubator.apache.org>,
<us...@storm.apache.org>
Subject: Storm DRPC Performance
Hi,
Am doing a feasibility as to using the storm DRPC to power our backend APIs
and have run into some problems. The core functionality that needs to be
executed takes around 2 ms to execute but in storm takes close to 400 msec.
Requesting an approach for them -
1. The LinearDRPCTopologyBuilder uses DRPCSpout by default. Is there a way
to increase the no. of instances on them? I see that spout always has
executors at 1. The problem is that when more than one request comes in at
the same time, the times go up drastically.
2. The bolts are defined as bolt0, bolt1, bolt2 etc. I used only 3 bolts but
there are 5 bolts shown (bolt0 to bolt4). Is there a way I can change the
names of the bolts shown in storm UI to more meaningful way the way it
happens in regular storm?
3. For some of the bolts, the Process Latency is very very high compared to
execute latency. I am trying to figure out the reason but am unsuccessful.
4. The Complete Latency on the spout is very high at 500+ msc. I am trying
to figure out the reason but am unsuccessful.
Configurations used as below: :
config.put(Config.DRPC_WORKER_THREADS,64);
config.setNumWorkers(50);
config.setMaxTaskParallelism(50);
config.setMaxSpoutPending(5000);
config.put(Config.TOPOLOGY_ACKER_EXECUTORS, 50);
config.put(Config.TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE, 16384);
config.put(Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE, 16384);
Supervisor slot ports: [6700 6701 6702 6703 6704 6705 6706 6707 6708 6709
6710 6711]
I have cluster of 5 servers with 8 GB and 4 cores. Of the 5, 3 servers have
DRPC running.
Storm UI Details:
Please do help me out.
Regards,
Kashyap