[GitHub] [beam] damccorm opened a new issue, #21032: Instructions for running Beam Python on Flink do not work on Mac

[GitBox <gi...@apache.org>]

damccorm opened a new issue, #21032:
URL: https://github.com/apache/beam/issues/21032

   I am trying to run Beam on a separate Flink cluster, as described here: [https://beam.apache.org/documentation/runners/flink/#executing-a-beam-pipeline-on-a-flink-cluster](https://beam.apache.org/documentation/runners/flink/#executing-a-beam-pipeline-on-a-flink-cluster)
   
   ??(1) Start a Flink cluster which exposes the Rest interface (e.g. `localhost:8081` by default).??
   
   Flink 1.13.1 starts successfully on localhost:8081.
   
   ??(2) Start JobService with Flink Rest endpoint: `docker run --net=host apache/beam_flink1.10_job_server:latest --flink-master=localhost:8081`.??
   
   I am running this instead, which again starts successfully:
   
   `docker run --net=host apache/beam_flink1.13_job_server:latest --flink-master=localhost:8081`
   
   ??(3) Submit the pipeline as above.??
   
   I'm running the WordCount pipeline as:
   
   `python -m apache_beam.examples.wordcount --input /Users/stefan/datastore/input.txt`
   ` --output /Users/stefan/datastore/output.txt`
   ` --runner=PortableRunner --job_endpoint=localhost:8099`
   
   After 1 minute the pipeline crashes with a `grpc.FutureTimeoutError`. This is the full output:
   
    
   
   `WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.`
    `INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.31.0`
    `INFO:root:No image given, using default Python SDK image`
    `WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.`
    `INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.31.0`
    `INFO:root:Python SDK container image set to "apache/beam_python3.8_sdk:2.31.0" for Docker environment`
    `INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x137d441f0> ====================`
    `INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x137d44280> ====================`
    `INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x137d449d0> ====================`
    `Traceback (most recent call last):`
    \` File "/usr/local/Cellar/python@3.8/3.8.11/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main`
    \` return _run_code(code, main_globals, None,`
    \` File "/usr/local/Cellar/python@3.8/3.8.11/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code`
    \` exec(code, run_globals)`
    \` File "/Users/stefan/workspace/biodiversity/.env/lib/python3.8/site-packages/apache_beam/examples/wordcount.py", line 94, in <module>`
    \` run()`
    \` File "/Users/stefan/workspace/biodiversity/.env/lib/python3.8/site-packages/apache_beam/examples/wordcount.py", line 89, in run`
    \` output | 'Write' >> WriteToText(known_args.output)`
    ` File "/Users/stefan/workspace/biodiversity/.env/lib/python3.8/site-packages/apache_beam/pipeline.py", line 585, in __exit__`
    \` self.result = self.run()`
    \` File "/Users/stefan/workspace/biodiversity/.env/lib/python3.8/site-packages/apache_beam/pipeline.py", line 564, in run`
    \` return self.runner.run_pipeline(self, self._options)`
    \` File "/Users/stefan/workspace/biodiversity/.env/lib/python3.8/site-packages/apache_beam/runners/portability/portable_runner.py", line 438, in run_pipeline`
    \` job_service_handle = self.create_job_service(options)`
    \` File "/Users/stefan/workspace/biodiversity/.env/lib/python3.8/site-packages/apache_beam/runners/portability/portable_runner.py", line 317, in create_job_service`
    \` return self.create_job_service_handle(server.start(), options)`
    \` File "/Users/stefan/workspace/biodiversity/.env/lib/python3.8/site-packages/apache_beam/runners/portability/job_server.py", line 54, in start`
    \` grpc.channel_ready_future(channel).result(timeout=self._timeout)`
    \` File "/Users/stefan/workspace/biodiversity/.env/lib/python3.8/site-packages/grpc/_utilities.py", line 140, in result`
    \` self._block(timeout)`
    \` File "/Users/stefan/workspace/biodiversity/.env/lib/python3.8/site-packages/grpc/_utilities.py", line 86, in _block`
    \` raise grpc.FutureTimeoutError()`
    `grpc.FutureTimeoutError`
   
    
   
   What's going on? Any help would be appreciated.
   
   Imported from Jira [BEAM-12657](https://issues.apache.org/jira/browse/BEAM-12657). Original Jira may contain additional context.
   Reported by: stefanistrate.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org