You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by tz...@apache.org on 2020/06/02 09:16:22 UTC
[flink-statefun] branch master updated (a854d16 -> 8376afa)
This is an automated email from the ASF dual-hosted git repository.
tzulitai pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink-statefun.git.
from a854d16 [hotfix] [legal] Add missing junixsocket-native-common entry in distribution NOTICE
new ae04d0a [FLINK-18015] [doc] Add documentation for state expiration
new 8376afa [FLINK-18016] [doc] Add documentation for UDS domain socket transport
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/sdk/java.md | 36 ++++++++++++++++++++++++++++++++++++
docs/sdk/modules.md | 12 +++++++++---
docs/sdk/python.md | 5 ++---
3 files changed, 47 insertions(+), 6 deletions(-)
[flink-statefun] 01/02: [FLINK-18015] [doc] Add documentation for
state expiration
Posted by tz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tzulitai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-statefun.git
commit ae04d0ad62a27d125417934b5e4eacb8410c94e8
Author: Tzu-Li (Gordon) Tai <tz...@apache.org>
AuthorDate: Mon Jun 1 16:04:13 2020 +0800
[FLINK-18015] [doc] Add documentation for state expiration
This closes #123.
---
docs/sdk/java.md | 36 ++++++++++++++++++++++++++++++++++++
docs/sdk/modules.md | 9 ++++++---
docs/sdk/python.md | 5 ++---
3 files changed, 44 insertions(+), 6 deletions(-)
diff --git a/docs/sdk/java.md b/docs/sdk/java.md
index e321944..05e7192 100644
--- a/docs/sdk/java.md
+++ b/docs/sdk/java.md
@@ -364,6 +364,42 @@ PersistedTable<String, Integer> table = PersistedTable.of("my-table", String.cla
PersistedAppendingBuffer<Integer> buffer = PersistedAppendingBuffer.of("my-buffer", Integer.class);
{% endhighlight %}
+### State Expiration
+
+Persisted states may be configured to expire and be deleted after a specified duration.
+This is supported by all types of state:
+
+{% highlight java %}
+@Persisted
+PersistedValue<Integer> table = PersistedValue.of(
+ "my-value",
+ Integer.class,
+ Expiration.expireAfterWriting(Duration.ofHours(1)));
+
+@Persisted
+PersistedTable<String, Integer> table = PersistedTable.of(
+ "my-table",
+ String.class,
+ Integer.class,
+ Expiration.expireAfterWriting(Duration.ofMinutes(5)));
+
+@Persisted
+PersistedAppendingBuffer<Integer> buffer = PersistedAppendingBuffer.of(
+ "my-buffer",
+ Integer.class,
+ Expiration.expireAfterWriting(Duration.ofSeconds(30)));
+{% endhighlight %}
+
+There are two expiration modes supported:
+
+{% highlight java %}
+Expiration.expireAfterWriting(...)
+
+Expiration.expireAfterReadingOrWriting(...)
+{% endhighlight %}
+
+State TTL configurations are made fault-tolerant by the runtime. In the case of downtime, state entries that should have been removed during said downtime will be purged immediately on restart.
+
## Function Providers and Dependency Injection
Stateful functions are created across a distributed cluster of nodes.
diff --git a/docs/sdk/modules.md b/docs/sdk/modules.md
index 320dbb1..c2ad041 100644
--- a/docs/sdk/modules.md
+++ b/docs/sdk/modules.md
@@ -94,7 +94,9 @@ A ``function`` is described via a number of properties.
* ``function.spec.endpoint``
* The endpoint at which the function is reachable.
* ``function.spec.states``
- * A list of the names of the persisted values decalred within the remote function.
+ * A list of the persisted values declared within the remote function.
+ * Each entry consists of a `name` property and an optional `expireAfter` property.
+ * Default for `expireAfter` - 0, meaning that state expiration is disabled.
* ``function.spec.maxNumBatchRequests``
* The maximum number of records that can be processed by a function for a particular ``address`` before invoking backpressure on the system.
* Default - 1000
@@ -105,7 +107,7 @@ A ``function`` is described via a number of properties.
### Full Example
{% highlight yaml %}
-version: "1.0"
+version: "2.0"
module:
meta:
@@ -119,7 +121,8 @@ module:
spec:
endpoint: http://<host-name>/statefun
states:
- - seen_count
+ - name: seen_count
+ expireAfter: 5min
maxNumBatchRequests: 500
timeout: 2min
{% endhighlight %}
diff --git a/docs/sdk/python.md b/docs/sdk/python.md
index d9eaefc..4cfd1e3 100644
--- a/docs/sdk/python.md
+++ b/docs/sdk/python.md
@@ -158,9 +158,8 @@ All stateful functions may contain state by merely storing values within the ``c
The data is always scoped to a specific function type and identifier.
State values could be absent, ``None``, or a ``google.protobuf.Any``.
-<div class="alert alert-info">
- <strong>Attention:</strong> [Remote modules]({{ site.baseurl}}/sdk/modules.html#remote-module) require that all state values are eagerly registered at module.yaml.
-</div>
+<strong>Attention:</strong> [Remote modules]({{ site.baseurl}}/sdk/modules.html#remote-module) require that all state values are eagerly registered at module.yaml.
+It'll also allow configuring other state properties, such as state expiration. Please refer to that page for more details.
Below is a stateful function that greets users based on the number of times they have been seen.
[flink-statefun] 02/02: [FLINK-18016] [doc] Add documentation for
UDS domain socket transport
Posted by tz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tzulitai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-statefun.git
commit 8376afa6b064bfa2374eefbda5e149cd490985c0
Author: Tzu-Li (Gordon) Tai <tz...@apache.org>
AuthorDate: Mon Jun 1 16:14:50 2020 +0800
[FLINK-18016] [doc] Add documentation for UDS domain socket transport
This closes #124.
---
docs/sdk/modules.md | 3 +++
1 file changed, 3 insertions(+)
diff --git a/docs/sdk/modules.md b/docs/sdk/modules.md
index c2ad041..5a33482 100644
--- a/docs/sdk/modules.md
+++ b/docs/sdk/modules.md
@@ -93,6 +93,9 @@ A ``function`` is described via a number of properties.
* The function type, defined as ``<namespace>/<name>``.
* ``function.spec.endpoint``
* The endpoint at which the function is reachable.
+ * Supported schemes are: ``http``, ``https``.
+ * Transport via UNIX domain sockets is supported by using the schemes ``http+unix`` or ``https+unix``.
+ * When using UNIX domain sockets, the endpoint format is: ``http+unix://<socket-file-path>/<serve-url-path>``. For example, ``http+unix:///uds.sock/path/of/url``.
* ``function.spec.states``
* A list of the persisted values declared within the remote function.
* Each entry consists of a `name` property and an optional `expireAfter` property.