[jira] [Created] (SAMZA-953) User-friendly error when a message with an empty key is sent to a log compacted Kafka topic

["Jake Maes (JIRA)" <ji...@apache.org>]

Jake Maes created SAMZA-953:
-------------------------------

             Summary: User-friendly error when a message with an empty key is sent to a log compacted Kafka topic
                 Key: SAMZA-953
                 URL: https://issues.apache.org/jira/browse/SAMZA-953
             Project: Samza
          Issue Type: Bug
            Reporter: Jake Maes


While verifying the 10.1 release with a test job, I started getting a CORRUPT_MESSAGE exception from Kafka while flushing the changelog. 

Turns out, the job was writing an empty string as a Key in the store. After serialization, the string translated to an empty byte[], which is not allowed for log compacted topics. 

The error message isn't intuitive for users. Ideally we'd catch the issue in the KV store, but the key hasn't been serialized at that point. The next best place to catch it is right after serialization, where we can measure the key size and throw if sending to a log compacted topic.

We could add this check in the current KafkaSystemProducer, but there is discussion about moving to the high-level kafka system producer which, I believe, performs serialization. We should coordinate with the kafka team to see if they have a more useful error message in the high-level producer. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)