[jira] [Created] (KAFKA-2517) Performance Regression post SSL implementation

["Ben Stopford (JIRA)" <ji...@apache.org>]

Ben Stopford created KAFKA-2517:
-----------------------------------

             Summary: Performance Regression post SSL implementation
                 Key: KAFKA-2517
                 URL: https://issues.apache.org/jira/browse/KAFKA-2517
             Project: Kafka
          Issue Type: Bug
            Reporter: Ben Stopford
            Assignee: Ben Stopford
            Priority: Critical
             Fix For: 0.8.3


It would appear that we incurred a performance regression on submission of the SSL work affecting the performance of the new Kafka Consumer. 

Running with 1KB messages. Macbook 2.3 GHz Intel Core i7, 8GB, APPLE SSD SM256E. Single server instance. All local. 

kafka-consumer-perf-test.sh ... --messages 3000000  --new-consumer

Pre-SSL changes (commit 503bd36647695e8cc91893ffb80346dd03eb0bc5)
Steady state throughputs = 234.8 MB/s
(2861.5913, 234.8261, 3000596, 246233.0543)

Post-SSL changes (commit 13c432f7952de27e9bf8cb4adb33a91ae3a4b738) 
Steady state throughput =  178.1 MB/s  
(2861.5913, 178.1480, 3000596, 186801.7182)

Implication is a 25% reduction in consumer throughput for these test conditions. 

This appears to be caused by the use of PlaintextTransportLayer rather than SocketChannel in FileMessageSet.writeTo() meaning a zero copy transfer is not invoked.

Switching to the use of a SocketChannel directly in FileMessageSet.writeTo()  yields the following result:
Steady state throughput =  281.8 MB/s
(2861.5913, 281.8191, 3000596, 295508.7650)





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)