You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Jiayi Zhao (JIRA)" <ji...@apache.org> on 2019/04/01 17:43:00 UTC

[jira] [Created] (BEAM-6956) --experiments=worker_threads=100 issue

Jiayi Zhao created BEAM-6956:
--------------------------------

             Summary: --experiments=worker_threads=100 issue
                 Key: BEAM-6956
                 URL: https://issues.apache.org/jira/browse/BEAM-6956
             Project: Beam
          Issue Type: New Feature
          Components: runner-flink
            Reporter: Jiayi Zhao


I noticed that without this "–experiments=worker_threads=100", pipeline will stuck,

The weird thing is, I tried some complex pipeline using the in thread flink method (./gradlew :beam-runners-flink_2.11-job-server:runShadow)

"–experiments=worker_threads=100" doesn't work, but "–experiments=worker_threads=1000" works fine

Then I tried the same pipeline using the separate local flink cluster (./gradlew :beam-runners-flink_2.11-job-server:runShadow -PflinkMasterUrl=localhost:8081), flink version is 1.5.6 (other version doesn't work, see BEAM-6915)

Neither did "–experiments=worker_threads=1000" or "–experiments=worker_threads=10000" work, pipeline stuck at certain stage (shows running in flink UI but won't finish forever)

any real fix to that? Thanks!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)