You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by Vojtech Janota <vo...@gmail.com> on 2018/08/30 08:36:23 UTC

[BEAM-960] Backoff in the DirectRunner if no work is available

Hi beamers,

I would like to contribute fix for the following issue:

   - https://issues.apache.org/jira/browse/BEAM-690

The corresponding PR:

   - https://github.com/apache/beam/pull/6303

I tried to follow the approach suggested in the comments of the said ticket
and any feedback is appreciated. This is my first attempt to contribute to
Beam code, so please bear with me if I missed something important.

To give you some background on why I'm fixing this:

I understand that performance related issues in direct runner generally
receive low priority as the whole direct runner concept is not meant for a
production deployment. Yet I think that this issue should receive
reasonable attention because even if being used in testing/CI pipelines,
the increased CPU consumption may materialise in a form of higher bill from
your favourite cloud provider. This definitely is our case and it makes
this issue a high priority one for us.

Regards,
Vojta

Re: [BEAM-960] Backoff in the DirectRunner if no work is available

Posted by Ismaël Mejía <ie...@gmail.com>.

Hello Vojtech,

Thanks a lot for your work on this fix and for pinging us for review.
Improving all runners is important and CPU improvements in direct
runner definitely are welcome because we all benefit of this (for
testing).

Review is in process so you should have news soon.

On Thu, Aug 30, 2018 at 10:36 AM Vojtech Janota <vo...@gmail.com> wrote:
>
> Hi beamers,
>
> I would like to contribute fix for the following issue:
>
> https://issues.apache.org/jira/browse/BEAM-690
>
> The corresponding PR:
>
> https://github.com/apache/beam/pull/6303
>
> I tried to follow the approach suggested in the comments of the said ticket and any feedback is appreciated. This is my first attempt to contribute to Beam code, so please bear with me if I missed something important.
>
> To give you some background on why I'm fixing this:
>
> I understand that performance related issues in direct runner generally receive low priority as the whole direct runner concept is not meant for a production deployment. Yet I think that this issue should receive reasonable attention because even if being used in testing/CI pipelines, the increased CPU consumption may materialise in a form of higher bill from your favourite cloud provider. This definitely is our case and it makes this issue a high priority one for us.
>
> Regards,
> Vojta