You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Shanthoosh Venkataraman (JIRA)" <ji...@apache.org> on 2018/07/30 22:15:00 UTC

[jira] [Created] (SAMZA-1793) Upgrade to recent API for kafka consumer / producer.

Shanthoosh Venkataraman created SAMZA-1793:
----------------------------------------------

             Summary: Upgrade to recent API for kafka consumer / producer.
                 Key: SAMZA-1793
                 URL: https://issues.apache.org/jira/browse/SAMZA-1793
             Project: Samza
          Issue Type: New Feature
            Reporter: Shanthoosh Venkataraman
            Assignee: Shanthoosh Venkataraman


Kafka consumer: 
1.  Samza currently uses SimpleConsumer to consume messages from the KafkaBroker. However, the SimpleConsumer API is deprecated and is slated to be removed in future kafka-clients release. It's essential to migrate to KafkaConsumer api sooner to continue kafka connector in samza. 

2. Internally within linkedIn, samza framework uses KafkaConsumer api. There's a ongoing effort for building release tests within linkedin. By having uniformity of API usage between OSS and internal linkedin branches, it will be easy to port internal release tests to open source.

Kafka Producer: 

Since 0.10.1, Kafka’s messages can carry timestamps, indicating the time the event has occurred or the time when the message has been written to the Kafka broker. This is essential to support rewinding of checkpoints based upon time.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)