[jira] [Work logged] (BEAM-9049) MemoryMonitor thrashing detection is too aggressive for batch workers

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-9049?focusedWorklogId=367186&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367186 ]

ASF GitHub Bot logged work on BEAM-9049:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 07/Jan/20 02:43
            Start Date: 07/Jan/20 02:43
    Worklog Time Spent: 10m 
      Work Description: TheNeuralBit commented on issue #10499: [BEAM-9049] Add opt-in --shutDownOnThrashing flag
URL: https://github.com/apache/beam/pull/10499#issuecomment-571408052
 
 
   Closing this as I think we're better off making changes just for the batch worker.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 367186)
    Time Spent: 1.5h  (was: 1h 20m)

> MemoryMonitor thrashing detection is too aggressive for batch workers
> ---------------------------------------------------------------------
>
>                 Key: BEAM-9049
>                 URL: https://issues.apache.org/jira/browse/BEAM-9049
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-dataflow
>            Reporter: Brian Hulette
>            Assignee: Brian Hulette
>            Priority: Major
>             Fix For: 2.19.0
>
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> In the streaming dataflow worker we've implemented push-back so that we will reduce parallelism when there is memory pressure. Since we cannot do this on the batch worker, it doesn't make sense for our thrashing detection to be so aggressive. We should increase the thresholds used for thrashing detection when running on Batch workloads.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)