[GitHub] [beam] piotr-szuberski edited a comment on pull request #12297: [BEAM-10137] Add KinesisIO.Read external transform

[GitBox <gi...@apache.org>]

piotr-szuberski edited a comment on pull request #12297:
URL: https://github.com/apache/beam/pull/12297#issuecomment-664541514


   @TheNeuralBit I've moved all commits to this PR. I have a feeling that big PR is better than few smaller PRs dependent of each other - they are much less maintainable. I'm used to Gerrit where each dependent commit is a PR rebased on the previous commit.
   
   I tried to use testcontainers with localstack but I encountered a problem that I can create, read and write streams via boto3, but I'm unable to use it with Beam. What I found is that kinesis java sdk by default uses CBOR which can be disabled by executing java program with kinesis sdk with executing it with -Dcom.amazonaws.sdk.disableCertChecking=true. But I don't have an idea how to do that in beam's execution model - do you have any idea or suggestion how could I do that?
   
   In general the tests before introducing localstack work correctly. Commits after the localstack introduction don't work, I just put vanilla kinesis test to indicate that the container seems to work properly.
   
   Edit: write test is now working after adding disabling certificate verification option.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org