You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Peter Kleinmann <nn...@gmail.com> on 2022/07/05 18:32:47 UTC

multi-language beam sdk base image for multi-language pipelines in locked down envs? [python] [java] [beam sdk] [docker] [multi-language]

Hi Community,

Sorry, I am fairly new at this, so my question might be a bit incoherent,
but here goes -

I'm at big company X, and we've determined we want to have
a) python developers writing Dataflow pipelines,
b) occasionally calling out to SQLTransform, but let's also imagine
CustomJavaTransformY with dependenicies J,K
c) we want to deploy this in a highly limited VPC with no access to PYPI or
Maven, or the internet at large

So, how can we get this to work, given we are starting from within python,
and have limited connectivity?