[jira] [Created] (BEAM-2704) KafkaIO: NPE without key serializer set

["Chris Pettitt (JIRA)" <ji...@apache.org>]

Chris Pettitt created BEAM-2704:
-----------------------------------

             Summary: KafkaIO: NPE without key serializer set
                 Key: BEAM-2704
                 URL: https://issues.apache.org/jira/browse/BEAM-2704
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-extensions
            Reporter: Chris Pettitt
            Assignee: Davor Bonaci


The KafkaIO javadoc implies that you do not need to set a Serializer if you only want to emit values:

{code}
 * <p>Often you might want to write just values without any keys to Kafka. Use {@code values()} to
 * write records with default empty(null) key:
 *
 * <pre>{@code
 *  PCollection<String> strings = ...;
 *  strings.apply(KafkaIO.<Void, String>write()
 *      .withBootstrapServers("broker_1:9092,broker_2:9092")
 *      .withTopic("results")
 *      .withValueSerializer(new StringSerializer()) // just need serializer for value
 *      .values()
 *    );
 * }</pre>
{code}

However, if you don't set the key serializer then Kafka blows up when trying to instantiate the key serializer (in Kafka 0.10.1, at least). It would be more convenient if KafkaIO worked as documented and assigned a null serializer if values() is used.  

Relevant stack trace:

{code}
Caused by: java.lang.NullPointerException
	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:230)
	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:163)
	at org.apache.beam.sdk.io.kafka.KafkaIO$KafkaWriter.setup(KafkaIO.java:1582)
	at org.apache.beam.sdk.io.kafka.KafkaIO$KafkaWriter$DoFnInvoker.invokeSetup(Unknown Source)
{code}





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)