You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/05/15 12:48:31 UTC
[GitHub] [flink-statefun] igalshilman opened a new pull request #113: [Flink-17644] Add state expiration to the Java SDK
igalshilman opened a new pull request #113:
URL: https://github.com/apache/flink-statefun/pull/113
This PR adds state expiration (state TTL in Flink land) to the persisted states in the Java SDK. follow up PR would expose that to the model.yaml.
There are two types of expirations:
• expiration after the last write
* expiration after the last write or read
The change was verified manually by modifying the Greeter example to
expire it’s state 5 seconds after writing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-statefun] sjwiesman commented on a change in pull request #113: [Flink-17644] Add state expiration to the Java SDK
Posted by GitBox <gi...@apache.org>.
sjwiesman commented on a change in pull request #113:
URL: https://github.com/apache/flink-statefun/pull/113#discussion_r426813593
##########
File path: statefun-sdk/src/main/java/org/apache/flink/statefun/sdk/state/Expiration.java
##########
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.statefun.sdk.state;
+
+import java.time.Duration;
+import java.util.Objects;
+
+/**
+ * State Expiration Configuration
+ *
+ * <p>This class defines the way state can be auto expired by the runtime. State expiration (also
+ * known as state TTL) can be used to keep state from growing arbitrarily by assigning an expiration
+ * date to a {@link PersistedAppendingBuffer}, {@link PersistedValue} or a {@link PersistedTable}.
+ *
+ * <p>State can be expired after a duration had passed since either from the last write to the
+ * state, or the last read.
+ */
+public final class Expiration {
+ public enum Mode {
+ NONE,
+ AFTER_WRITE,
+ AFTER_READ_OR_WRITE;
+ }
+
+ /**
+ * Returns an Expiration configuration that would expire a @duration after the last write.
+ *
+ * @param duration a duration to wait before considering the state expired.
+ */
+ public static Expiration expireAfterWriting(Duration duration) {
+ return new Expiration(Mode.AFTER_WRITE, duration);
+ }
+
+ /**
+ * Returns an Expiration configuration that would expire a @duration after the last write or read.
+ *
+ * @param duration a duration to wait before considering the state expired.
+ */
+ public static Expiration expireAfterReadingOrWriting(Duration duration) {
+ return new Expiration(Mode.AFTER_READ_OR_WRITE, duration);
+ }
+
+ public static Expiration expireAfter(Duration duration, Mode mode) {
+ return new Expiration(mode, duration);
+ }
+
+ /** @return Returns a disabled expiration */
+ public static Expiration none() {
+ return new Expiration(Mode.NONE, Duration.ZERO);
+ }
+
+ private final Mode mode;
+ private final Duration duration;
+
+ public Expiration(Mode mode, Duration duration) {
Review comment:
What about annotating with @ForRuntime to keep it public but limit the surface area?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-statefun] sjwiesman closed pull request #113: [Flink-17644] Add state expiration to the Java SDK
Posted by GitBox <gi...@apache.org>.
sjwiesman closed pull request #113:
URL: https://github.com/apache/flink-statefun/pull/113
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-statefun] sjwiesman commented on a change in pull request #113: [Flink-17644] Add state expiration to the Java SDK
Posted by GitBox <gi...@apache.org>.
sjwiesman commented on a change in pull request #113:
URL: https://github.com/apache/flink-statefun/pull/113#discussion_r426782885
##########
File path: statefun-sdk/src/main/java/org/apache/flink/statefun/sdk/state/Expiration.java
##########
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.statefun.sdk.state;
+
+import java.time.Duration;
+import java.util.Objects;
+
+/**
+ * State Expiration Configuration
+ *
+ * <p>This class defines the way state can be auto expired by the runtime. State expiration (also
+ * known as state TTL) can be used to keep state from growing arbitrarily by assigning an expiration
+ * date to a {@link PersistedAppendingBuffer}, {@link PersistedValue} or a {@link PersistedTable}.
+ *
+ * <p>State can be expired after a duration had passed since either from the last write to the
+ * state, or the last read.
+ */
+public final class Expiration {
+ public enum Mode {
+ NONE,
+ AFTER_WRITE,
+ AFTER_READ_OR_WRITE;
+ }
+
+ /**
+ * Returns an Expiration configuration that would expire a @duration after the last write.
+ *
+ * @param duration a duration to wait before considering the state expired.
+ */
+ public static Expiration expireAfterWriting(Duration duration) {
+ return new Expiration(Mode.AFTER_WRITE, duration);
+ }
+
+ /**
+ * Returns an Expiration configuration that would expire a @duration after the last write or read.
+ *
+ * @param duration a duration to wait before considering the state expired.
+ */
+ public static Expiration expireAfterReadingOrWriting(Duration duration) {
+ return new Expiration(Mode.AFTER_READ_OR_WRITE, duration);
+ }
+
+ public static Expiration expireAfter(Duration duration, Mode mode) {
+ return new Expiration(mode, duration);
+ }
+
+ /** @return Returns a disabled expiration */
+ public static Expiration none() {
+ return new Expiration(Mode.NONE, Duration.ZERO);
+ }
+
+ private final Mode mode;
+ private final Duration duration;
+
+ public Expiration(Mode mode, Duration duration) {
Review comment:
?
```suggestion
private Expiration(Mode mode, Duration duration) {
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-statefun] sjwiesman commented on a change in pull request #113: [Flink-17644] Add state expiration to the Java SDK
Posted by GitBox <gi...@apache.org>.
sjwiesman commented on a change in pull request #113:
URL: https://github.com/apache/flink-statefun/pull/113#discussion_r426858040
##########
File path: statefun-sdk/src/main/java/org/apache/flink/statefun/sdk/state/Expiration.java
##########
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.statefun.sdk.state;
+
+import java.time.Duration;
+import java.util.Objects;
+
+/**
+ * State Expiration Configuration
+ *
+ * <p>This class defines the way state can be auto expired by the runtime. State expiration (also
+ * known as state TTL) can be used to keep state from growing arbitrarily by assigning an expiration
+ * date to a {@link PersistedAppendingBuffer}, {@link PersistedValue} or a {@link PersistedTable}.
+ *
+ * <p>State can be expired after a duration had passed since either from the last write to the
+ * state, or the last read.
+ */
+public final class Expiration {
+ public enum Mode {
+ NONE,
+ AFTER_WRITE,
+ AFTER_READ_OR_WRITE;
+ }
+
+ /**
+ * Returns an Expiration configuration that would expire a @duration after the last write.
+ *
+ * @param duration a duration to wait before considering the state expired.
+ */
+ public static Expiration expireAfterWriting(Duration duration) {
+ return new Expiration(Mode.AFTER_WRITE, duration);
+ }
+
+ /**
+ * Returns an Expiration configuration that would expire a @duration after the last write or read.
+ *
+ * @param duration a duration to wait before considering the state expired.
+ */
+ public static Expiration expireAfterReadingOrWriting(Duration duration) {
+ return new Expiration(Mode.AFTER_READ_OR_WRITE, duration);
+ }
+
+ public static Expiration expireAfter(Duration duration, Mode mode) {
+ return new Expiration(mode, duration);
+ }
+
+ /** @return Returns a disabled expiration */
+ public static Expiration none() {
+ return new Expiration(Mode.NONE, Duration.ZERO);
+ }
+
+ private final Mode mode;
+ private final Duration duration;
+
+ public Expiration(Mode mode, Duration duration) {
Review comment:
After an offline discussion we settled on the annotation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-statefun] sjwiesman commented on pull request #113: [Flink-17644] Add state expiration to the Java SDK
Posted by GitBox <gi...@apache.org>.
sjwiesman commented on pull request #113:
URL: https://github.com/apache/flink-statefun/pull/113#issuecomment-630400906
Closed in dd85b1184715a4e19aac527584e00f9e17c47aeb
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-statefun] igalshilman commented on a change in pull request #113: [Flink-17644] Add state expiration to the Java SDK
Posted by GitBox <gi...@apache.org>.
igalshilman commented on a change in pull request #113:
URL: https://github.com/apache/flink-statefun/pull/113#discussion_r426786392
##########
File path: statefun-sdk/src/main/java/org/apache/flink/statefun/sdk/state/Expiration.java
##########
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.statefun.sdk.state;
+
+import java.time.Duration;
+import java.util.Objects;
+
+/**
+ * State Expiration Configuration
+ *
+ * <p>This class defines the way state can be auto expired by the runtime. State expiration (also
+ * known as state TTL) can be used to keep state from growing arbitrarily by assigning an expiration
+ * date to a {@link PersistedAppendingBuffer}, {@link PersistedValue} or a {@link PersistedTable}.
+ *
+ * <p>State can be expired after a duration had passed since either from the last write to the
+ * state, or the last read.
+ */
+public final class Expiration {
+ public enum Mode {
+ NONE,
+ AFTER_WRITE,
+ AFTER_READ_OR_WRITE;
+ }
+
+ /**
+ * Returns an Expiration configuration that would expire a @duration after the last write.
+ *
+ * @param duration a duration to wait before considering the state expired.
+ */
+ public static Expiration expireAfterWriting(Duration duration) {
+ return new Expiration(Mode.AFTER_WRITE, duration);
+ }
+
+ /**
+ * Returns an Expiration configuration that would expire a @duration after the last write or read.
+ *
+ * @param duration a duration to wait before considering the state expired.
+ */
+ public static Expiration expireAfterReadingOrWriting(Duration duration) {
+ return new Expiration(Mode.AFTER_READ_OR_WRITE, duration);
+ }
+
+ public static Expiration expireAfter(Duration duration, Mode mode) {
+ return new Expiration(mode, duration);
+ }
+
+ /** @return Returns a disabled expiration */
+ public static Expiration none() {
+ return new Expiration(Mode.NONE, Duration.ZERO);
+ }
+
+ private final Mode mode;
+ private final Duration duration;
+
+ public Expiration(Mode mode, Duration duration) {
Review comment:
The public constructor would be useful for the JsonModule.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org