[flink-statefun] 01/02: [FLINK-18015] [doc] Add documentation for state expiration

[tz...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

tzulitai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-statefun.git

commit ae04d0ad62a27d125417934b5e4eacb8410c94e8
Author: Tzu-Li (Gordon) Tai <tz...@apache.org>
AuthorDate: Mon Jun 1 16:04:13 2020 +0800

    [FLINK-18015] [doc] Add documentation for state expiration
    
    This closes #123.
---
 docs/sdk/java.md    | 36 ++++++++++++++++++++++++++++++++++++
 docs/sdk/modules.md |  9 ++++++---
 docs/sdk/python.md  |  5 ++---
 3 files changed, 44 insertions(+), 6 deletions(-)

diff --git a/docs/sdk/java.md b/docs/sdk/java.md
index e321944..05e7192 100644
--- a/docs/sdk/java.md
+++ b/docs/sdk/java.md
@@ -364,6 +364,42 @@ PersistedTable<String, Integer> table = PersistedTable.of("my-table", String.cla
 PersistedAppendingBuffer<Integer> buffer = PersistedAppendingBuffer.of("my-buffer", Integer.class);
 {% endhighlight %}
 
+### State Expiration
+
+Persisted states may be configured to expire and be deleted after a specified duration.
+This is supported by all types of state:
+
+{% highlight java %}
+@Persisted
+PersistedValue<Integer> table = PersistedValue.of(
+    "my-value",
+    Integer.class,
+    Expiration.expireAfterWriting(Duration.ofHours(1)));
+
+@Persisted
+PersistedTable<String, Integer> table = PersistedTable.of(
+    "my-table",
+    String.class,
+    Integer.class,
+    Expiration.expireAfterWriting(Duration.ofMinutes(5)));
+
+@Persisted
+PersistedAppendingBuffer<Integer> buffer = PersistedAppendingBuffer.of(
+    "my-buffer",
+    Integer.class,
+    Expiration.expireAfterWriting(Duration.ofSeconds(30)));
+{% endhighlight %}
+
+There are two expiration modes supported:
+
+{% highlight java %}
+Expiration.expireAfterWriting(...)
+
+Expiration.expireAfterReadingOrWriting(...)
+{% endhighlight %}
+
+State TTL configurations are made fault-tolerant by the runtime. In the case of downtime, state entries that should have been removed during said downtime will be purged immediately on restart.
+
 ## Function Providers and Dependency Injection
 
 Stateful functions are created across a distributed cluster of nodes.
diff --git a/docs/sdk/modules.md b/docs/sdk/modules.md
index 320dbb1..c2ad041 100644
--- a/docs/sdk/modules.md
+++ b/docs/sdk/modules.md
@@ -94,7 +94,9 @@ A ``function`` is described via a number of properties.
 * ``function.spec.endpoint``
     * The endpoint at which the function is reachable.
 * ``function.spec.states``
-    * A list of the names of the persisted values decalred within the remote function.
+    * A list of the persisted values declared within the remote function.
+    * Each entry consists of a `name` property and an optional `expireAfter` property.
+    * Default for `expireAfter` - 0, meaning that state expiration is disabled.
 * ``function.spec.maxNumBatchRequests``
     * The maximum number of records that can be processed by a function for a particular ``address`` before invoking backpressure on the system.
     * Default - 1000
@@ -105,7 +107,7 @@ A ``function`` is described via a number of properties.
 ### Full Example
 
 {% highlight yaml %}
-version: "1.0"
+version: "2.0"
 
 module:
   meta:
@@ -119,7 +121,8 @@ module:
         spec:
           endpoint: http://<host-name>/statefun
           states:
-            - seen_count
+            - name: seen_count
+              expireAfter: 5min
           maxNumBatchRequests: 500
           timeout: 2min
 {% endhighlight %}
diff --git a/docs/sdk/python.md b/docs/sdk/python.md
index d9eaefc..4cfd1e3 100644
--- a/docs/sdk/python.md
+++ b/docs/sdk/python.md
@@ -158,9 +158,8 @@ All stateful functions may contain state by merely storing values within the ``c
 The data is always scoped to a specific function type and identifier.
 State values could be absent, ``None``, or a ``google.protobuf.Any``.
 
-<div class="alert alert-info">
-  <strong>Attention:</strong> [Remote modules]({{ site.baseurl}}/sdk/modules.html#remote-module) require that all state values are eagerly registered at module.yaml.
-</div>
+<strong>Attention:</strong> [Remote modules]({{ site.baseurl}}/sdk/modules.html#remote-module) require that all state values are eagerly registered at module.yaml.
+It'll also allow configuring other state properties, such as state expiration. Please refer to that page for more details.
     
 Below is a stateful function that greets users based on the number of times they have been seen.