You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/11 11:25:40 UTC
[GitHub] [pulsar] amitkaushal16 opened a new issue #11639: Reduction in Broker TPS on 2.8 Pulsar build
amitkaushal16 opened a new issue #11639:
URL: https://github.com/apache/pulsar/issues/11639
Hi All,
###
We have observed a reduction in Broker TPS on 2.8 builds compares to 2.6 build against the same test.
<meta charset="utf-8"><b style="font-weight:normal;" id="docs-internal-guid-a498c25e-7fff-ac42-d9be-e9d1ad5d84ef"><h1 dir="ltr" style="line-height:1.38;margin-top:24pt;margin-bottom:6pt;"><span style="font-size:23pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Environment Setup</span></h1><p dir="ltr" style="line-height:1.38;margin-top:12pt;margin-bottom:12pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Pulsar is setup in AWS account with following infrastructure configuration.</span></p><p dir="ltr" style="line-height:1.38;margin-top:12pt;margin-bottom:12pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;
font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Note: If any changes done in infrastructure for a specific test are mentioned under performance test case details</span></p><div dir="ltr" style="margin-left:0pt;" align="left">
Component | Type | #Instances
-- | -- | --
Broker | c5.4xlarge | 3
bookie | m5.xlarge | 6
zookeeper | t2.large | 3 (One additional zk in east-1)
proxy | c5.2xlarge | [3]([url]([url](url)))
manager | c5.2xlarge | 1
</div></b>
Config Customisations on build
Message(transparent) Encryption enabled
Logging enabled for Publish, Dispatch, ACK events.
TLS enabled over transport among all components.
Strict Authentication and authorization enabled at the proxy
managedLedgerDefaultEnsembleSize=2
managedLedgerDefaultWriteQuorum=2
managedLedgerDefaultAckQuorum=2
Pulsar 2.6 Perf test result
Swimlane capacity at ~60% CPU for 3KB msg size - Achieved TPS - 58k
3 proxies+3 zk+ 1 zk in east-1 6 bookies per region
3 Scripts, Each script has 3 instances, Topics:25, 7 inject producer users, 7 inject consumer users, 1 producer connections,1 consumer connections
Pulsar 2.8 Perf test result
Swimlane capacity at ~35% CPU for 3KB msg size - Achieved TPS - 37K
3 proxies+3 zk+ 1 zk in east-1 6 bookies per region
3 Scripts, Each script has 3 instances, Topics:25, 7 inject producer users, 7 inject consumer users, 1 producer connections,1 consumer connections
Observation :
Beyond one point broker TPS was not increasing beyond 35%.
We are suspecting there is some throttle value in broker configuration which is limiting broker CPU and [TPS.]([url]([url]([url]([url]([url](url))))))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #11639: Reduction in Broker TPS on 2.8 Pulsar build
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #11639:
URL: https://github.com/apache/pulsar/issues/11639#issuecomment-1058887188
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org