Re: [I] [Bug] Beam errors out when using PortableRunner (Flink Runner) – `Cannot run program "docker"` [beam]

["felipecaputo (via GitHub)" <gi...@apache.org>]

felipecaputo commented on issue #30663:
URL: https://github.com/apache/beam/issues/30663#issuecomment-2221201609

   I'm currently facing the same problem myself, using a beam yaml pipeline, when it comes to run the SQL transformation it intents to run a docker with java in `flink-main-container`
   
   ```
   WARN  org.apache.beam.runners.fnexecution.environment.DockerCommand [] - Unable to pull docker image apache/beam_java17_sdk:2.56.0, cause: Cannot run program "docker": error=2, No such file or directory
   ERROR org.apache.flink.runtime.operators.BatchTask                 [] - Error in task code:  CHAIN MapPartition (MapPartition at [3]Sql/BeamCoGBKJoinRel_77/Join.Impl/CoGroup.ExpandCrossProduct/{extractKeylhs, CoGroupByKey}) -> FlatMap (FlatMap at ExtractOutput[0]) (1/1)
   java.lang.Exception: The user defined 'open()' method caused an exception: java.io.IOException: Cannot run program "docker": error=2, No such file or directory
   org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:508) ~[flink-dist-1.17.2.jar:1.17.2]
   org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:357) ~[flink-dist-1.17.2.jar:1.17.2]
   org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) [flink-dist-1.17.2.jar:1.17.2]
   org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) [flink-dist-1.17.2.jar:1.17.2]
   org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) [flink-dist-1.17.2.jar:1.17.2]
   org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) [flink-dist-1.17.2.jar:1.17.2]
   java.lang.Thread.run(Unknown Source) [?:?]
   ```
   
   The tasks were executed in the `python-harness` container runned smoothly, but when it come to the Sql transformation, it failed
   <img width="256" alt="image" src="https://github.com/apache/beam/assets/7751449/2bf9318f-c930-487b-8778-160a8bf852a7">
   <img width="697" alt="image" src="https://github.com/apache/beam/assets/7751449/7204ca82-c2c1-49d5-a218-5dbdb7a771b8">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org