Need some pointers to writing (real) tests

[Jaikiran Pai <ja...@gmail.com>]

I have been looking at some unassigned JIRAs to work on during some 
spare time and found this one 
https://issues.apache.org/jira/browse/KAFKA-1837. As I note in that 
JIRA, I can see why this happens and have a potential fix for it. But to 
first reproduce the issue and then verify the fix, I have been 
attempting a testcase (in the clients). Some of the tests that are 
already present (like SenderTest) use MockProducer which won't be 
relevant in testing this issue, from what I see.

So I need some inputs or pointers to create a test which will use the 
real KafkaProducer/Sender/NetworkClient. My initial attempt at this uses 
the TestUtils to create a (dummy) cluster, and that one fails for 
obvious reasons (the client not receiving a metadata update over the 
wire from the server):

     @Test
     public void testFailedSend() throws Exception {
         final TopicPartition tp = new TopicPartition("test", 0);
         final String producedValue = "foobar";
         final ProducerRecord product = new ProducerRecord(tp.topic(), 
producedValue);
         final Cluster cluster = TestUtils.singletonCluster("test", 1);
         final Node node = this.cluster.nodes().get(0);
         final Properties kakfaProducerConfigs = new Properties();
kakfaProducerConfigs.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, 
node.host() + ":" + node.port());
kakfaProducerConfigs.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, 
StringSerializer.class.getName());
kakfaProducerConfigs.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, 
StringSerializer.class.getName());
         final Producer producer = new KafkaProducer(kakfaProducerConfigs);

         // This times out waiting for a metadata update from the server 
for the cluster (because there isn't really any real server around)
         final Future<RecordMetadata> futureAck = producer.send(product);
         ....


Any pointers to existing tests?

-Jaikiran

Re: Need some pointers to writing (real) tests

[Jay Kreps <ja...@confluent.io>]

The integration tests for the producer are in with the server since the
dependency is that the server depends on the clients rather than vice
versa). Only the mock tests are with the clients. You should be able to add
to one of the tests in
  core/src/test/scala/integration/kafka/api/Producer*


On Wed, Jan 14, 2015 at 11:07 PM, Jaikiran Pai <ja...@gmail.com>
wrote:

> I have been looking at some unassigned JIRAs to work on during some spare
> time and found this one https://issues.apache.org/jira/browse/KAFKA-1837.
> As I note in that JIRA, I can see why this happens and have a potential fix
> for it. But to first reproduce the issue and then verify the fix, I have
> been attempting a testcase (in the clients). Some of the tests that are
> already present (like SenderTest) use MockProducer which won't be relevant
> in testing this issue, from what I see.
>
> So I need some inputs or pointers to create a test which will use the real
> KafkaProducer/Sender/NetworkClient. My initial attempt at this uses the
> TestUtils to create a (dummy) cluster, and that one fails for obvious
> reasons (the client not receiving a metadata update over the wire from the
> server):
>
>     @Test
>     public void testFailedSend() throws Exception {
>         final TopicPartition tp = new TopicPartition("test", 0);
>         final String producedValue = "foobar";
>         final ProducerRecord product = new ProducerRecord(tp.topic(),
> producedValue);
>         final Cluster cluster = TestUtils.singletonCluster("test", 1);
>         final Node node = this.cluster.nodes().get(0);
>         final Properties kakfaProducerConfigs = new Properties();
> kakfaProducerConfigs.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
> node.host() + ":" + node.port());
> kakfaProducerConfigs.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
> StringSerializer.class.getName());
> kakfaProducerConfigs.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
> StringSerializer.class.getName());
>         final Producer producer = new KafkaProducer(kakfaProducerConfigs);
>
>         // This times out waiting for a metadata update from the server
> for the cluster (because there isn't really any real server around)
>         final Future<RecordMetadata> futureAck = producer.send(product);
>         ....
>
>
> Any pointers to existing tests?
>
> -Jaikiran
>