You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by ja...@apache.org on 2020/03/19 05:34:33 UTC
[flink] 01/07: [hotfix][doc] Fix typos and broken links in memory
configuration docs.
This is an automated email from the ASF dual-hosted git repository.
jark pushed a commit to branch release-1.10
in repository https://gitbox.apache.org/repos/asf/flink.git
commit edc092d7eae4769dcde29b233e871005707a10f9
Author: Xintong Song <to...@gmail.com>
AuthorDate: Wed Mar 11 15:23:19 2020 +0800
[hotfix][doc] Fix typos and broken links in memory configuration docs.
---
docs/ops/memory/mem_migration.md | 6 +++---
docs/ops/memory/mem_migration.zh.md | 6 +++---
docs/ops/memory/mem_setup.md | 8 ++++----
docs/ops/memory/mem_setup.zh.md | 8 ++++----
docs/ops/memory/mem_trouble.md | 8 ++++----
docs/ops/memory/mem_trouble.zh.md | 8 ++++----
docs/ops/memory/mem_tuning.md | 6 +++---
docs/ops/memory/mem_tuning.zh.md | 6 +++---
8 files changed, 28 insertions(+), 28 deletions(-)
diff --git a/docs/ops/memory/mem_migration.md b/docs/ops/memory/mem_migration.md
index 46a3f2e..7fd219e 100644
--- a/docs/ops/memory/mem_migration.md
+++ b/docs/ops/memory/mem_migration.md
@@ -123,7 +123,7 @@ The following options are deprecated but if they are still used they will be int
Although, the network memory configuration has not changed too much it is recommended to verify its configuration.
It can change if other memory components have new sizes, e.g. the total memory which the network can be a fraction of.
-See also [new detailed memory model](mem_setup.html#detailed-memory-model).
+See also [new detailed memory model](mem_detail.html).
The container cut-off configuration options, [`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio)
and [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min), have no effect for task manager processes anymore
@@ -158,7 +158,7 @@ Additionally, you can now have more direct control over the JVM heap assigned to
([`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)),
see also [Task (Operator) Heap Memory](mem_setup.html#task-operator-heap-memory).
The JVM heap memory is also used by the heap state backends ([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
-or [FsStateBackend](../state/state_backends.html#the-fsstatebackend) if it is chosen for streaming jobs.
+or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)) if it is chosen for streaming jobs.
A part of the JVM heap is now always reserved for Flink framework
([`taskmanager.memory.framework.heap.size`](../config.html#taskmanager-memory-framework-heap-size)).
@@ -217,7 +217,7 @@ The other direct or native off-heap memory consumers can now be addressed by the
* Task off-heap memory ([`taskmanager.memory.task.off-heap.size`](../config.html#taskmanager-memory-task-off-heap-size))
* Framework off-heap memory ([`taskmanager.memory.framework.off-heap.size`](../config.html#taskmanager-memory-framework-off-heap-size))
* JVM metaspace ([`taskmanager.memory.jvm-metaspace.size`](../config.html#taskmanager-memory-jvm-metaspace-size))
-* JVM overhead (see also [detailed new memory model](mem_setup.html#detailed-memory-model))
+* JVM overhead (see also [detailed new memory model](mem_detail.html))
<span class="label label-info">Note</span> The job manager still has container cut-off memory configuration options.
The mentioned configuration options remain valid for the job manager in the same way as before.
diff --git a/docs/ops/memory/mem_migration.zh.md b/docs/ops/memory/mem_migration.zh.md
index 46a3f2e..7fd219e 100644
--- a/docs/ops/memory/mem_migration.zh.md
+++ b/docs/ops/memory/mem_migration.zh.md
@@ -123,7 +123,7 @@ The following options are deprecated but if they are still used they will be int
Although, the network memory configuration has not changed too much it is recommended to verify its configuration.
It can change if other memory components have new sizes, e.g. the total memory which the network can be a fraction of.
-See also [new detailed memory model](mem_setup.html#detailed-memory-model).
+See also [new detailed memory model](mem_detail.html).
The container cut-off configuration options, [`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio)
and [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min), have no effect for task manager processes anymore
@@ -158,7 +158,7 @@ Additionally, you can now have more direct control over the JVM heap assigned to
([`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)),
see also [Task (Operator) Heap Memory](mem_setup.html#task-operator-heap-memory).
The JVM heap memory is also used by the heap state backends ([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
-or [FsStateBackend](../state/state_backends.html#the-fsstatebackend) if it is chosen for streaming jobs.
+or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)) if it is chosen for streaming jobs.
A part of the JVM heap is now always reserved for Flink framework
([`taskmanager.memory.framework.heap.size`](../config.html#taskmanager-memory-framework-heap-size)).
@@ -217,7 +217,7 @@ The other direct or native off-heap memory consumers can now be addressed by the
* Task off-heap memory ([`taskmanager.memory.task.off-heap.size`](../config.html#taskmanager-memory-task-off-heap-size))
* Framework off-heap memory ([`taskmanager.memory.framework.off-heap.size`](../config.html#taskmanager-memory-framework-off-heap-size))
* JVM metaspace ([`taskmanager.memory.jvm-metaspace.size`](../config.html#taskmanager-memory-jvm-metaspace-size))
-* JVM overhead (see also [detailed new memory model](mem_setup.html#detailed-memory-model))
+* JVM overhead (see also [detailed new memory model](mem_detail.html))
<span class="label label-info">Note</span> The job manager still has container cut-off memory configuration options.
The mentioned configuration options remain valid for the job manager in the same way as before.
diff --git a/docs/ops/memory/mem_setup.md b/docs/ops/memory/mem_setup.md
index 7e27ee2..32cfd9c 100644
--- a/docs/ops/memory/mem_setup.md
+++ b/docs/ops/memory/mem_setup.md
@@ -47,7 +47,7 @@ and by the JVM to run the process. The *total Flink memory* consumption includes
</center>
<br />
-If you run FIink locally (e.g. from your IDE) without creating a cluster, then only a subset of the memory configuration
+If you run Flink locally (e.g. from your IDE) without creating a cluster, then only a subset of the memory configuration
options are relevant, see also [local execution](mem_detail.html#local-execution) for more details.
Otherwise, the simplest way to setup memory in Flink is to configure either of the two following options:
@@ -55,7 +55,7 @@ Otherwise, the simplest way to setup memory in Flink is to configure either of t
* Total process memory ([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size))
The rest of the memory components will be adjusted automatically, based on default values or additionally configured options.
-[Here](mem_detail.html#detailed-memory-model) are more details about the other memory components.
+[Here](mem_detail.html) are more details about the other memory components.
Configuring *total Flink memory* is better suited for standalone deployments where you want to declare how much memory
is given to Flink itself. The *total Flink memory* splits up into JVM heap, [managed memory size](#managed-memory)
@@ -87,7 +87,7 @@ to specify explicitly both [task heap](#task-operator-heap-memory) and [managed
It gives more control over the available JVM heap to Flink’s tasks and its [managed memory](#managed-memory).
The rest of the memory components will be adjusted automatically, based on default values or additionally configured options.
-[Here](mem_detail.html#detailed-memory-model) are more details about the other memory components.
+[Here](mem_detail.html) are more details about the other memory components.
<span class="label label-info">Note</span> If you have configured the task heap and managed memory explicitly, it is recommended to set neither
*total process memory* nor *total Flink memory*. Otherwise, it may easily lead to memory configuration conflicts.
@@ -130,4 +130,4 @@ see also [JVM parameters](mem_detail.html#jvm-parameters).
<span class="label label-info">Note</span> The *network memory* is also part of JVM *direct memory* but it is managed by Flink and guaranteed
to never exceed its configured size. Therefore, resizing the *network memory* will not help in this situation.
-See also [the detailed memory model](mem_detail.html#detailed-memory-model).
+See also [the detailed memory model](mem_detail.html).
diff --git a/docs/ops/memory/mem_setup.zh.md b/docs/ops/memory/mem_setup.zh.md
index 7e27ee2..32cfd9c 100644
--- a/docs/ops/memory/mem_setup.zh.md
+++ b/docs/ops/memory/mem_setup.zh.md
@@ -47,7 +47,7 @@ and by the JVM to run the process. The *total Flink memory* consumption includes
</center>
<br />
-If you run FIink locally (e.g. from your IDE) without creating a cluster, then only a subset of the memory configuration
+If you run Flink locally (e.g. from your IDE) without creating a cluster, then only a subset of the memory configuration
options are relevant, see also [local execution](mem_detail.html#local-execution) for more details.
Otherwise, the simplest way to setup memory in Flink is to configure either of the two following options:
@@ -55,7 +55,7 @@ Otherwise, the simplest way to setup memory in Flink is to configure either of t
* Total process memory ([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size))
The rest of the memory components will be adjusted automatically, based on default values or additionally configured options.
-[Here](mem_detail.html#detailed-memory-model) are more details about the other memory components.
+[Here](mem_detail.html) are more details about the other memory components.
Configuring *total Flink memory* is better suited for standalone deployments where you want to declare how much memory
is given to Flink itself. The *total Flink memory* splits up into JVM heap, [managed memory size](#managed-memory)
@@ -87,7 +87,7 @@ to specify explicitly both [task heap](#task-operator-heap-memory) and [managed
It gives more control over the available JVM heap to Flink’s tasks and its [managed memory](#managed-memory).
The rest of the memory components will be adjusted automatically, based on default values or additionally configured options.
-[Here](mem_detail.html#detailed-memory-model) are more details about the other memory components.
+[Here](mem_detail.html) are more details about the other memory components.
<span class="label label-info">Note</span> If you have configured the task heap and managed memory explicitly, it is recommended to set neither
*total process memory* nor *total Flink memory*. Otherwise, it may easily lead to memory configuration conflicts.
@@ -130,4 +130,4 @@ see also [JVM parameters](mem_detail.html#jvm-parameters).
<span class="label label-info">Note</span> The *network memory* is also part of JVM *direct memory* but it is managed by Flink and guaranteed
to never exceed its configured size. Therefore, resizing the *network memory* will not help in this situation.
-See also [the detailed memory model](mem_detail.html#detailed-memory-model).
+See also [the detailed memory model](mem_detail.html).
diff --git a/docs/ops/memory/mem_trouble.md b/docs/ops/memory/mem_trouble.md
index cd6463b..f3e6b13 100644
--- a/docs/ops/memory/mem_trouble.md
+++ b/docs/ops/memory/mem_trouble.md
@@ -29,7 +29,7 @@ under the License.
If you see an *IllegalConfigurationException* thrown from *TaskExecutorProcessUtils*, it usually indicates
that there is either an invalid configuration value (e.g. negative memory size, fraction that is greater than 1, etc.)
-or configuration conflicts. Check the documentation chapters related to the [memory components](mem_setup.html#detailed-memory-model)
+or configuration conflicts. Check the documentation chapters related to the [memory components](mem_detail.html)
mentioned in the exception message.
## OutOfMemoryError: Java heap space
@@ -44,7 +44,7 @@ is advanced and should only be changed if you are sure that the Flink framework
The exception usually indicates that the JVM *direct memory* limit is too small or that there is a *direct memory leak*.
Check whether user code or other external dependencies use the JVM *direct memory* and that it is properly accounted for.
-You can try to increase its limit by adjusting [direct off-heap memory](mem_setup.html#detailed-memory-model).
+You can try to increase its limit by adjusting [direct off-heap memory](mem_detail.html).
See also [how to configure off-heap memory](mem_setup.html#configure-off-heap-memory-direct-or-native) and
the [JVM arguments](mem_detail.html#jvm-parameters) which Flink sets.
@@ -55,7 +55,7 @@ You can try to increase the [JVM metaspace option](../config.html#taskmanager-me
## IOException: Insufficient number of network buffers
-The exception usually indicates that the size of the configured [network memory](mem_setup.html#detailed-memory-model)
+The exception usually indicates that the size of the configured [network memory](mem_detail.html)
is not big enough. You can try to increase the *network memory* by adjusting the following options:
* [`taskmanager.memory.network.min`](../config.html#taskmanager-memory-network-min)
* [`taskmanager.memory.network.max`](../config.html#taskmanager-memory-network-max)
@@ -70,5 +70,5 @@ monitoring system or from the error messages when a container gets killed by the
If [RocksDBStateBackend](../state/state_backends.html#the-rocksdbstatebackend) is used and the memory controlling is disabled,
you can try to increase the [managed memory](mem_setup.html#managed-memory).
-Alternatively, you can increase the [JVM overhead](mem_setup.html#detailed-memory-model).
+Alternatively, you can increase the [JVM overhead](mem_detail.html).
See also [how to configure memory for containers](mem_tuning.html#configure-memory-for-containers).
diff --git a/docs/ops/memory/mem_trouble.zh.md b/docs/ops/memory/mem_trouble.zh.md
index cd6463b..f3e6b13 100644
--- a/docs/ops/memory/mem_trouble.zh.md
+++ b/docs/ops/memory/mem_trouble.zh.md
@@ -29,7 +29,7 @@ under the License.
If you see an *IllegalConfigurationException* thrown from *TaskExecutorProcessUtils*, it usually indicates
that there is either an invalid configuration value (e.g. negative memory size, fraction that is greater than 1, etc.)
-or configuration conflicts. Check the documentation chapters related to the [memory components](mem_setup.html#detailed-memory-model)
+or configuration conflicts. Check the documentation chapters related to the [memory components](mem_detail.html)
mentioned in the exception message.
## OutOfMemoryError: Java heap space
@@ -44,7 +44,7 @@ is advanced and should only be changed if you are sure that the Flink framework
The exception usually indicates that the JVM *direct memory* limit is too small or that there is a *direct memory leak*.
Check whether user code or other external dependencies use the JVM *direct memory* and that it is properly accounted for.
-You can try to increase its limit by adjusting [direct off-heap memory](mem_setup.html#detailed-memory-model).
+You can try to increase its limit by adjusting [direct off-heap memory](mem_detail.html).
See also [how to configure off-heap memory](mem_setup.html#configure-off-heap-memory-direct-or-native) and
the [JVM arguments](mem_detail.html#jvm-parameters) which Flink sets.
@@ -55,7 +55,7 @@ You can try to increase the [JVM metaspace option](../config.html#taskmanager-me
## IOException: Insufficient number of network buffers
-The exception usually indicates that the size of the configured [network memory](mem_setup.html#detailed-memory-model)
+The exception usually indicates that the size of the configured [network memory](mem_detail.html)
is not big enough. You can try to increase the *network memory* by adjusting the following options:
* [`taskmanager.memory.network.min`](../config.html#taskmanager-memory-network-min)
* [`taskmanager.memory.network.max`](../config.html#taskmanager-memory-network-max)
@@ -70,5 +70,5 @@ monitoring system or from the error messages when a container gets killed by the
If [RocksDBStateBackend](../state/state_backends.html#the-rocksdbstatebackend) is used and the memory controlling is disabled,
you can try to increase the [managed memory](mem_setup.html#managed-memory).
-Alternatively, you can increase the [JVM overhead](mem_setup.html#detailed-memory-model).
+Alternatively, you can increase the [JVM overhead](mem_detail.html).
See also [how to configure memory for containers](mem_tuning.html#configure-memory-for-containers).
diff --git a/docs/ops/memory/mem_tuning.md b/docs/ops/memory/mem_tuning.md
index 9d83d00..de54513 100644
--- a/docs/ops/memory/mem_tuning.md
+++ b/docs/ops/memory/mem_tuning.md
@@ -31,7 +31,7 @@ depending on the use case and which options are important in which case.
## Configure memory for standalone deployment
It is recommended to configure [total Flink memory](mem_setup.html#configure-total-memory)
-([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)) or its [components](mem_setup.html#detailed-memory-model)
+([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)) or its [components](mem_detail.html)
for [standalone deployment](../deployment/cluster_setup.html) where you want to declare how much memory is given to Flink itself.
Additionally, you can adjust *JVM metaspace* if it causes [problems](mem_trouble.html#outofmemoryerror-metaspace).
@@ -47,7 +47,7 @@ It declares how much memory in total should be assigned to the Flink *JVM proces
<span class="label label-info">Note</span> If you configure the *total Flink memory* Flink will implicitly add JVM memory components
to derive the *total process memory* and request a container with the memory of that derived size,
-see also [detailed Memory Model](mem_setup.html#detailed-memory-model).
+see also [detailed Memory Model](mem_detail.html).
<div class="alert alert-warning">
<strong>Warning:</strong> If Flink or user code allocates unmanaged off-heap (native) memory beyond the container size
@@ -63,7 +63,7 @@ will dictate the optimal memory configurations of your cluster.
### Heap state backend
When running a stateless job or using a heap state backend ([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
-or [FsStateBackend](../state/state_backends.html#the-fsstatebackend), set [managed memory](mem_setup.html#managed-memory) to zero.
+or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)), set [managed memory](mem_setup.html#managed-memory) to zero.
This will ensure that the maximum amount of memory is allocated for user code on the JVM.
### RocksDB state backend
diff --git a/docs/ops/memory/mem_tuning.zh.md b/docs/ops/memory/mem_tuning.zh.md
index 9d83d00..de54513 100644
--- a/docs/ops/memory/mem_tuning.zh.md
+++ b/docs/ops/memory/mem_tuning.zh.md
@@ -31,7 +31,7 @@ depending on the use case and which options are important in which case.
## Configure memory for standalone deployment
It is recommended to configure [total Flink memory](mem_setup.html#configure-total-memory)
-([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)) or its [components](mem_setup.html#detailed-memory-model)
+([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)) or its [components](mem_detail.html)
for [standalone deployment](../deployment/cluster_setup.html) where you want to declare how much memory is given to Flink itself.
Additionally, you can adjust *JVM metaspace* if it causes [problems](mem_trouble.html#outofmemoryerror-metaspace).
@@ -47,7 +47,7 @@ It declares how much memory in total should be assigned to the Flink *JVM proces
<span class="label label-info">Note</span> If you configure the *total Flink memory* Flink will implicitly add JVM memory components
to derive the *total process memory* and request a container with the memory of that derived size,
-see also [detailed Memory Model](mem_setup.html#detailed-memory-model).
+see also [detailed Memory Model](mem_detail.html).
<div class="alert alert-warning">
<strong>Warning:</strong> If Flink or user code allocates unmanaged off-heap (native) memory beyond the container size
@@ -63,7 +63,7 @@ will dictate the optimal memory configurations of your cluster.
### Heap state backend
When running a stateless job or using a heap state backend ([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
-or [FsStateBackend](../state/state_backends.html#the-fsstatebackend), set [managed memory](mem_setup.html#managed-memory) to zero.
+or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)), set [managed memory](mem_setup.html#managed-memory) to zero.
This will ensure that the maximum amount of memory is allocated for user code on the JVM.
### RocksDB state backend