[GitHub] [pulsar] kuangye098 opened a new issue #4159: The maximum number of outstanding messages (--max-outstanding) configuration may be inaccurate when the topic is multi-partitioned

[GitBox <gi...@apache.org>]

kuangye098 opened a new issue #4159: The maximum number of outstanding messages (--max-outstanding) configuration may be inaccurate when the topic is multi-partitioned
URL: https://github.com/apache/pulsar/issues/4159
 
 
   In the Producer Performance Benchmark, Topic has a partition number of 1024, specifying the parameter [-o 20000], and found that the maximum number of unprocessed messages per producer is 48.
   ```shell
   02:23:10.036 [pulsar-client-io-2-1] INFO  org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {
     "topicName" : "persistent://xxxx/internal/benchmark2",
     "producerName" : null,
     "sendTimeoutMs" : 0,
     "blockIfQueueFull" : true,
     "maxPendingMessages" : 48,
     "maxPendingMessagesAcrossPartitions" : 50000,
     "messageRoutingMode" : "RoundRobinPartition",
     "hashingScheme" : "JavaStringHash",
     "cryptoFailureAction" : "FAIL",
     "batchingMaxPublishDelayMicros" : 1000,
     "batchingMaxMessages" : 1000,
     "batchingEnabled" : true,
     "compressionType" : "NONE",
     "initialSequenceId" : null,
     "autoUpdatePartitions" : true,
     "properties" : { }
   }
   ```
   
   1. Created a Topic with 1024 partitions.
   2. Test with the following command:
   ```shell
   /pulsar-perf produce persistent://xxxx/internal/benchmark2 -m 10000000 -n 10 -i 5 -time 600 -o 20000 -u pulsar://10.0.231.202:6650,10.0.219.35:6650,10.0.250.94:6650> 0428-002-producer.log
   ```
   **Expected**
   The number of outstanding consumption messages per partition is 20000 or 20000/partition number.
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services