[GitHub] [beam] damccorm opened a new issue, #20634: Unable to use KafkaIO with SSL via Python API

[GitBox <gi...@apache.org>]

damccorm opened a new issue, #20634:
URL: https://github.com/apache/beam/issues/20634

   I've come across the same issue described in BEAM-9627 while trying to use KafkaIO with SSL via the Python API.
   
   While the solution in the Java API is to use withConsumerFactoryFn or withProducerFactoryFn and provide a custom function to download or generate key/trust stores before the consumer or producer gets created, there is no such functionality exposed via in the Python API since it's just a wrapper around the external transform.
   
   Is there currently any other way to work around this issue in Python or would it be possible to expose a similar functionality in the API? (I'm using the Dataflow runner, but hopefully it shouldn't matter.)
   
   As far as my understanding goes, alternatively one could write an external transform that's a thin wrapper around the Java's KafkaIO API that handles key/trust stores under the hood and make it available via an external expansion service, but that's definitely way less ideal than being able to achieve the same thing via the built-in API.
   
   Imported from Jira [BEAM-11102](https://issues.apache.org/jira/browse/BEAM-11102). Original Jira may contain additional context.
   Reported by: mwz.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org