[jira] [Assigned] (SAMZA-111) SystemConsumers is slow with large partition count

["Chris Riccomini (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Riccomini reassigned SAMZA-111:
-------------------------------------

    Assignee: Chris Riccomini

> SystemConsumers is slow with large partition count
> --------------------------------------------------
>
>                 Key: SAMZA-111
>                 URL: https://issues.apache.org/jira/browse/SAMZA-111
>             Project: Samza
>          Issue Type: Bug
>          Components: container
>    Affects Versions: 0.6.0
>            Reporter: Chris Riccomini
>            Assignee: Chris Riccomini
>
> We have been seeing very slow processing speed when running a Samza container that consumes from 1000s of partitions. We don't see a corresponding slow speed when running the same code, but with fewer input partitions (say 8-24).
> The messages per second seems to drop off as more partitions are added to the Samza container. One Samza job has ~2500 partitions, and is seeing only 6000 messages/sec. The same code running with ~9 partitions is seeing 30,000 messages/sec.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)