[jira] [Updated] (SAMZA-126) BrokerProxy calls SimpleConsumer.fetch with empty request

["Chris Riccomini (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Riccomini updated SAMZA-126:
----------------------------------

    Attachment: SAMZA-126.0.patch

Attaching a fix. RB at https://reviews.apache.org/r/16763.

1. Added if statement to filter empty requests out.
2. Added a metric to count skipped fetches.
3. Added a sleep when we skip a fetch.
4. Added a test to verify changes.

Also ran changes through a real job on a YARN grid, and the metrics reflected the change.

> BrokerProxy calls SimpleConsumer.fetch with empty request
> ---------------------------------------------------------
>
>                 Key: SAMZA-126
>                 URL: https://issues.apache.org/jira/browse/SAMZA-126
>             Project: Samza
>          Issue Type: Bug
>          Components: kafka
>    Affects Versions: 0.6.0
>            Reporter: Chris Riccomini
>            Assignee: Chris Riccomini
>         Attachments: SAMZA-126.0.patch
>
>
> When I put a Thread.sleep(1000) in my process() method, I noticed that the reads metric in BrokerProxy:
>       metrics.brokerReads(host, port).inc
> Is still be incremented thousands of times a second. This is unexpected, since I should only be processing 1 message a second, and therefore not be fetching until the message buffer for my input stream is empty, again.
> Upon investigation, I found that fetchMessage will call simpleConsumer.fetch, even if the fetch request being issued has no topic partitions in it.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)