You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kyle Weaver (Jira)" <ji...@apache.org> on 2019/08/19 23:59:00 UTC

[jira] [Updated] (BEAM-7790) Make debugging subprocess workers easier

     [ https://issues.apache.org/jira/browse/BEAM-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kyle Weaver updated BEAM-7790:
------------------------------
    Description: 
[ ] The output of the SDK workers is currently invisible due to the output and logging setup.

[ ] The dockerized version of the Python SDK worker sets up an HTTP server to let the user view stack traces for all of the worker's threads [1]. It would be useful if this was available for other execution modes as well.

[x] BEAM-7676 Make the above items more usable with multiple subprocesses by identifying them with worker ids.

 

[1] [https://github.com/apache/beam/blob/9f4ce1c6fc2fb195e218783a6e9ce6104ddb4d1e/sdks/python/apache_beam/runners/worker/sdk_worker_main.py#L46-L89]

  was:
1. The output of the SDK workers is currently invisible due to the output and logging setup.

2. The dockerized version of the Python SDK worker sets up an HTTP server to let the user view stack traces for all of the worker's threads [1]. It would be useful if this was available for other execution modes as well.

3. Make the above items more usable with multiple subprocesses by identifying them with worker ids.

 

[1] [https://github.com/apache/beam/blob/9f4ce1c6fc2fb195e218783a6e9ce6104ddb4d1e/sdks/python/apache_beam/runners/worker/sdk_worker_main.py#L46-L89]


> Make debugging subprocess workers easier
> ----------------------------------------
>
>                 Key: BEAM-7790
>                 URL: https://issues.apache.org/jira/browse/BEAM-7790
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-py-harness
>            Reporter: Kyle Weaver
>            Assignee: Kyle Weaver
>            Priority: Minor
>
> [ ] The output of the SDK workers is currently invisible due to the output and logging setup.
> [ ] The dockerized version of the Python SDK worker sets up an HTTP server to let the user view stack traces for all of the worker's threads [1]. It would be useful if this was available for other execution modes as well.
> [x] BEAM-7676 Make the above items more usable with multiple subprocesses by identifying them with worker ids.
>  
> [1] [https://github.com/apache/beam/blob/9f4ce1c6fc2fb195e218783a6e9ce6104ddb4d1e/sdks/python/apache_beam/runners/worker/sdk_worker_main.py#L46-L89]



--
This message was sent by Atlassian Jira
(v8.3.2#803003)