[GitHub] [pulsar] sijie commented on issue #4199: documentation: unclear how to use Pulsar IO

[GitBox <gi...@apache.org>]

sijie commented on issue #4199: documentation: unclear how to use Pulsar IO
URL: https://github.com/apache/pulsar/issues/4199#issuecomment-489402850
 
 
   > Couldn't figure out from documentation how to use Pulsar IO connectors ((
   
   Thank you for your feedbacks. This helps us improve the documentation. We will try to make clarifications in the documentation according to your feedbacks.
   
   >  I couldn't find any information on how to configure SerDe for Pulsar IO? 
   
   Pulsar IO is more about getting data in and out. Most of the connectors just transferred `bytes`. Some of the connectors like CDC and JDBC will attempt to deserialize the events using pulsar generic schema.
   
   Currently Pulsar IO doesn't provide the ability to run `functions` along with the connectors. There are issues created for looking into that space.
   
   >  I would like to use external protobuf schemas:
   
   You can construct a schema instance using `Schema.PROTOBUF(<generated-protobuf-class>.class)`. Then you can use the schema instance in your applications to publish the protobuf messages to a pulsar topic. 
   
   Assume your generated protobuf class is `ProtobufClass`. Then you can write functions as followings:
   
   ```
   public class TestFunction implements Function<ProtobufClass, Void> {
   }
   ```
   
   when you submit such a function, you can specify `--schema-type PROTOBUF`. If you want to use your own SerDe, you can specify `--custom-serde-inputs`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services