Slack digest for #dev - 2020-03-12

[Apache Pulsar Slack <ap...@gmail.com>]

2020-03-11 09:57:39 UTC - Konstantinos Papalias: I'm coming into Pulsar from a Kafka / Kafka Streams background, and have been trying to adopt Pulsar / Functions for similar tasks, the main hurdles I see for Pulsar adoption, are the following based on my limited exposure:
1. The benefit of Kafka is not age, is the wide adoption across many companies and industries which brings a huge community around it in regards to forums / presentations / discussions / tooling and documentation. 
2. Kafka Streams is a really powerful tool to have in your arsenal that allows for advanced computations such as joins, aggregations, chaining functions, fan out, etc, while still running it as a library, no need to maintain a separate cluster (Spark / Flink) for simple enough computations. Whereas Pulsar Functions are still too limited in scope to be realistically practical. Or can be useful, but in a more limited scope. 
+1 : Eugen
----
2020-03-11 12:17:04 UTC - Konstantinos Papalias: Hello, how can someone enforce Pulsar Functions to use an Avro Schema for the output topic ? As far as I can see the default behaviour is still suffering from this bug <https://github.com/apache/pulsar/issues/3741> , how can I configure Avro Functions with `allowNull = false` ?
----
2020-03-11 14:14:51 UTC - Michael Kaufman: @Michael Kaufman has joined the channel
----
2020-03-11 16:26:24 UTC - Sijie Guo: Is your POJO generated from AVRO?
----
2020-03-11 16:27:06 UTC - Konstantinos Papalias: It's not I'm afraid, it's defined by hand
----
2020-03-11 16:32:38 UTC - Konstantinos Papalias: So if I deploy the function where the output topic has not defined a Schema, the Auto-generated Schema would be Nullable, if I predefine the Schema for the Topic, it throws an exception with incompatible Schema:
```ERROR o.a.pulsar.functions.sink.PulsarSink - Failed to create Producer while doing user publish
org.apache.pulsar.client.api.PulsarClientException$IncompatibleSchemaException: org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException: Incompatible schema used
	at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:252)
	at org.apache.pulsar.client.impl.ProducerBuilderImpl.create(ProducerBuilderImpl.java:88)
	at org.apache.pulsar.functions.sink.PulsarSink$PulsarSinkProcessorBase.createProducer(PulsarSink.java:104)
	at org.apache.pulsar.functions.sink.PulsarSink$PulsarSinkAtMostOnceProcessor.&lt;init&gt;(PulsarSink.java:172)
	at org.apache.pulsar.functions.sink.PulsarSink$PulsarSinkAtLeastOnceProcessor.&lt;init&gt;(PulsarSink.java:194)
	at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:267)
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:788)
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupJavaInstance(JavaInstanceRunnable.java:213)
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:244)
	at java.base/java.lang.Thread.run(Thread.java:834)```
----
2020-03-11 16:49:20 UTC - Sijie Guo: I see. If it is hand-wrote POJO, I am afraid that we cannot disable `allowNull` at this moment. Can you create a github issue about this?
ok_hand : Konstantinos Papalias
----
2020-03-11 16:50:19 UTC - Konstantinos Papalias: Sure np, I was afraid this might have been the case, but do you believe it would have worked if the POJO was generated from Schema ?
----