You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by StephanEwen <gi...@git.apache.org> on 2017/02/06 15:46:21 UTC

[GitHub] flink pull request #3276: [FLINK-5718] [core] TaskManagers exit the JVM on f...

GitHub user StephanEwen opened a pull request:

    https://github.com/apache/flink/pull/3276

    [FLINK-5718] [core] TaskManagers exit the JVM on fatal exceptions.

    *This adds a feature requested by a user for production stability.*
    
    Certain exceptions should not be attempted to be handled by the TaskManager, because they indicate that the JVM is corrupt. When the task throws such an exception, the TaskManager simply forcefully and immediately exits the JVM.
    
    Optionally, the `OutOfMemoryError` can also be set to cause such immediate JVM termination, via the `taskmanager.jvm-exit-on-oom` config option.
    
    
    ### Tests
    
    This adds a test that tests the option and the actual process kill (via a spawned test process). 
    
    ### Documentation
    
    This adds the `taskmanager.jvm-exit-on-oom` to the `setup/config.md` docs.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StephanEwen/incubator-flink exit_on_fatal_error

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/3276.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3276
    
----
commit 21c08817554e5a66186afa83158ca9c6ac975ba4
Author: Stephan Ewen <se...@apache.org>
Date:   2017-02-06T14:52:39Z

    [FLINK-5718] [core] TaskManagers exit the JVM on fatal exceptions.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #3276: [FLINK-5718] [core] TaskManagers exit the JVM on f...

Posted by StephanEwen <gi...@git.apache.org>.

Github user StephanEwen commented on a diff in the pull request:

https://github.com/apache/flink/pull/3276#discussion_r100128721

--- Diff: docs/setup/config.md ---
@@ -86,7 +86,7 @@ The default fraction for managed memory can be adjusted using the `taskmanager.m

- `taskmanager.memory.segment-size`: The size of memory buffers used by the memory manager and the network stack in bytes (DEFAULT: 32768 (= 32 KiBytes)).

-- `taskmanager.memory.preallocate`: Can be either of `true` or `false`. Specifies whether task managers should allocate all managed memory when starting up. (DEFAULT: false). When `taskmanager.memory.off-heap` is set to `true`, then it is advised that this configuration is also set to `true`. If this configuration is set to `false` cleaning up of the allocated offheap memory happens only when the configured JVM parameter MaxDirectMemorySize is reached by triggering a full GC.
+- `taskmanager.memory.preallocate`: Can be either of `true` or `false`. Specifies whether task managers should allocate all managed memory when starting up. (DEFAULT: false). When `taskmanager.memory.off-heap` is set to `true`, then it is advised that this configuration is also set to `true`. If this configuration is set to `false` cleaning up of the allocated offheap memory happens only when the configured JVM parameter MaxDirectMemorySize is reached by triggering a full GC. **Note:** For streaming setups, we highly recommend to set this value to `false` as the core state backends currently do not use the managed memory.
--- End diff --

That would probably be good

---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---