You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "RocMarshal (via GitHub)" <gi...@apache.org> on 2025/11/28 15:51:28 UTC
[PR] Flink 38340 [flink]
RocMarshal opened a new pull request, #27285:
URL: https://github.com/apache/flink/pull/27285
<!--
*Thank you very much for contributing to Apache Flink - we are happy that you want to help us improve Flink. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.*
*Please understand that we do not do this to make contributions to Flink a hassle. In order to uphold a high standard of quality for code contributions, while at the same time managing a large number of contributions, we need contributors to prepare the contributions well, and give reviewers enough contextual information for the review. Please also understand that contributions that do not follow this guide will take longer to review and thus typically be picked up with lower priority by the community.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [JIRA issue](https://issues.apache.org/jira/projects/FLINK/issues). Exceptions are made for typos in JavaDoc or documentation files, which need no JIRA issue.
- Name the pull request in the form "[FLINK-XXXX] [component] Title of the pull request", where *FLINK-XXXX* should be replaced by the actual issue number. Skip *component* if you are unsure about which is the best component.
Typo fixes that have no associated JIRA issue should be named following this pattern: `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Make sure that the change passes the automated tests, i.e., `mvn clean verify` passes. You can set up Azure Pipelines CI to do that following [this guide](https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-Tutorial:SettingupAzurePipelinesforaforkoftheFlinkrepository).
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message (including the JIRA id)
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
## What is the purpose of the change
[FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations
## Brief change log
[FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations
## Verifying this change
This change added tests and can be verified as follows:
- *RescalesSummaryTest.java*
- *DefaultRescaleTimelineTest.ajva*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774979581
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
Review Comment:
Thanks @davidradl
> Is this a getter or a create?
more a creator method.
And change the return type to `RescaleIdInfo`, the class has the attribute to represent the uuid of rescales.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728161951
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
Review Comment:
javadoc would be good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2732842121
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
Review Comment:
Thx @davidradl .
updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2841098225
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/DefaultRescaleTimeline.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+import org.apache.flink.runtime.util.BoundedFIFOQueue;
+import org.apache.flink.util.AbstractID;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.function.Supplier;
+
+/** Default implementation of {@link RescaleTimeline}. */
+public class DefaultRescaleTimeline implements RescaleTimeline {
+
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultRescaleTimeline.class);
+
+ private final Supplier<JobInformation> jobInformationGetter;
+
+ private RescaleIdInfo rescaleIdInfo;
+
+ @Nullable private Rescale currentRescale;
Review Comment:
@ferenc-csaky Thank you very much for your review.
I updated it based on your comments.
PTAL~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728201196
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescalesSummary.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.util.stats.StatsSummary;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.Serializable;
+
+/** Statistics summary of rescales. */
+public class RescalesSummary implements Serializable {
+ private static final long serialVersionUID = 1L;
+
+ private static final Logger LOG = LoggerFactory.getLogger(RescalesSummary.class);
+
+ private final StatsSummary allTerminatedSummary;
+
+ private final StatsSummary completedRescalesSummary;
+ private final StatsSummary ignoredRescalesSummary;
+ private final StatsSummary failedRescalesSummary;
+
+ // Total terminated count
+ private long totalRescalesCount = 0L;
+ private long completedRescalesCount = 0L;
+
+ private long ignoredRescalesCount = 0L;
+ private long failedRescalesCount = 0L;
+ private long inProgressRescalesCount = 0L;
+
+ public RescalesSummary(int maxHistorySize) {
+ this.allTerminatedSummary = new StatsSummary(maxHistorySize);
+ this.completedRescalesSummary = new StatsSummary(maxHistorySize);
+ this.ignoredRescalesSummary = new StatsSummary(maxHistorySize);
+ this.failedRescalesSummary = new StatsSummary(maxHistorySize);
+ }
+
+ /**
+ * Add a terminated rescale in. Note, The method could be called after calling {@link
+ * #addInProgress(Rescale)}.
+ *
+ * @param rescale the target terminated rescale.
+ */
+ public void addTerminated(Rescale rescale) {
+ if (!Rescale.isTerminated(rescale)) {
+ LOG.warn(
+ "Unexpected rescale: {}, which will be ignored when computing statistics.",
+ rescale);
+ return;
+ }
+
+ allTerminatedSummary.add(rescale.getDuration().toMillis());
+ inProgressRescalesCount = 0;
+
+ if (rescale.getTerminalState() == null) {
+ return;
+ }
+
+ switch (rescale.getTerminalState()) {
+ case FAILED:
+ failedRescalesSummary.add(rescale.getDuration().toMillis());
+ failedRescalesCount++;
+ break;
+ case COMPLETED:
+ completedRescalesSummary.add(rescale.getDuration().toMillis());
+ completedRescalesCount++;
+ break;
+ case IGNORED:
+ ignoredRescalesSummary.add(rescale.getDuration().toMillis());
+ ignoredRescalesCount++;
+ break;
+ default:
+ break;
+ }
+ }
+
+ /**
+ * Add an in-progress rescale in. Note, The method could be called before {@link
+ * #addTerminated(Rescale)}.
+ *
+ * @param rescale the target non-terminated rescale.
+ */
+ public void addInProgress(Rescale rescale) {
+ if (Rescale.isTerminated(rescale)) {
+ LOG.warn("Unexpected rescale: {}, which will be ignored.", rescale);
+ return;
Review Comment:
nit: maybe use `else` instead of `return`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2732871811
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newCurrentRescale(boolean newRescaleEpoch);
Review Comment:
> Do we need newCurrentRescale, can we have newRescale?
Sounds good.
> I am wondering whether this creation method would be more appropriate in a factory.
Perhaps it's not necessary to use Factory, as the logic is currently complex and there is no common part that needs to be abstracted.
Of course, I'd like to hear more ideas.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728190561
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newCurrentRescale(boolean newRescaleEpoch);
+
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>ture</code> if update successfully <code>false</code> else.
+ */
+ boolean updateCurrentRescale(RescaleUpdater rescaleUpdater);
Review Comment:
I am curious why we do not have an exception for the failure.
Is there a non-current rescale?
I wonder if the uuid should be returned here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2732850393
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
Review Comment:
Added the new comments to clarify the method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774979581
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
Review Comment:
Thanks @davidradl
> Is this a getter or a create?
more a creator method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774933207
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
Review Comment:
nit: what do we mean by container here? I suspect we do not want to mention this word - as it could be confusing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728185290
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newCurrentRescale(boolean newRescaleEpoch);
Review Comment:
Do we need `newCurrentRescale,` can we have `newRescale`?
I am wondering whether this creation method would be more appropriate in a factory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774910115
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>true</code> if there are no rescales to perform. <code>false</code> else.
+ */
+ boolean isIdling();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #isIdling()} must be
+ * true when creating a new current rescale. When there's a rescale in transition phase, we must
+ * seal the current resale before creating a new rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>true</code> if create successfully <code>false</code> else.
+ */
+ boolean newRescale(boolean newRescaleEpoch);
+
+ /** Get the latest rescale for the specified terminal state. */
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale. It only makes sense to update a rescale when there is an ongoing
+ * rescale that is in the process of transition states.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>true</code> if update successfully <code>false</code> else.
+ */
+ boolean updateRescale(RescaleUpdater rescaleUpdater);
Review Comment:
could this be a default method` boolean updateRescale(Rescale rescaleToUpdate, RescaleUpdater rescaleUpdater);`
that just calls _rescaleUpdater.update(rescaleToUpdate);_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774811469
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>ture</code> if there are no rescales to perform. <code>false</code> else.
Review Comment:
ture -> true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "ferenc-csaky (via GitHub)" <gi...@apache.org>.
ferenc-csaky commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2803457915
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a timeline used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
Review Comment:
I guess this is not a public interface, if so lets explicitly mark it as `@Internal`.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/DefaultRescaleTimeline.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+import org.apache.flink.runtime.util.BoundedFIFOQueue;
+import org.apache.flink.util.AbstractID;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.function.Supplier;
+
+/** Default implementation of {@link RescaleTimeline}. */
+public class DefaultRescaleTimeline implements RescaleTimeline {
+
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultRescaleTimeline.class);
+
+ private final Supplier<JobInformation> jobInformationGetter;
+
+ private RescaleIdInfo rescaleIdInfo;
+
+ @Nullable private Rescale currentRescale;
Review Comment:
nit: I'd put these after the final fields to make the grouping cleaner
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>true</code> if there are no rescales to perform. <code>false</code> else.
+ */
+ boolean isIdling();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #isIdling()} must be
+ * true when creating a new current rescale. When there's a rescale in transition phase, we must
+ * seal the current resale before creating a new rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>true</code> if create successfully <code>false</code> else.
+ */
+ boolean newRescale(boolean newRescaleEpoch);
+
+ /** Get the latest rescale for the specified terminal state. */
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale. It only makes sense to update a rescale when there is an ongoing
+ * rescale that is in the process of transition states.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>true</code> if update successfully <code>false</code> else.
+ */
+ boolean updateRescale(RescaleUpdater rescaleUpdater);
+
+ /** Rescale operation interface. */
+ interface RescaleUpdater {
+ void update(Rescale rescaleToUpdate);
+ }
+
+ /** Default implementation of {@link RescaleTimeline} without any actions. */
Review Comment:
I'd simplify this to `No-op implementation of {@link RescaleTimeline}.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728190561
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newCurrentRescale(boolean newRescaleEpoch);
+
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>ture</code> if update successfully <code>false</code> else.
+ */
+ boolean updateCurrentRescale(RescaleUpdater rescaleUpdater);
Review Comment:
I am curious why we do not have an exception for the failure.
Is there a non-current rescale?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2732850393
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
Review Comment:
Added the new comments to clarify the method.
> I was expecting the rescale uuid to be returned (or is this something not to be exposed?) and an exception if the rescale could not be created.
The uuid is not related the public scope, so it isn't be exposed here.
The feature is a slave feature, which would be better not interrupt the main process logic of flink jobs.
So, we'd like to keep non-exceptions-throwing here and just use true or false to represent the result of the current action.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774920208
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>ture</code> if there are no rescales to perform. <code>false</code> else.
+ */
+ boolean isIdling();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #isIdling()} must be
+ * true when creating a new current rescale. When there's a rescale in transition phase, we must
+ * seal the current resale before creating a new rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newRescale(boolean newRescaleEpoch);
+
+ /** Get the latest rescale for the specified terminal state. */
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale. It only makes sense to update a rescale when there is an ongoing
Review Comment:
Thanks @davidradl .
<img width="1364" height="743" alt="image" src="https://github.com/user-attachments/assets/95fa4b69-2855-4b81-affb-35cf85f65142" />
I try to update it , However, this might potentially make the interface more complex as shown above(This would likely require introducing two additional methods in the interface.).
In my limited reading, It's just that this interface is for internal use, with only two implementations: one is a default one with no functionality, and the other is the only one used for core logic calls. So keeping the simplest implementation seems maybe a good choice.
Of course, if you feel this change is necessary, I'd like to make the update.
Please let me know WDYTA it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774959831
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/DefaultRescaleTimeline.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+import org.apache.flink.runtime.util.BoundedFIFOQueue;
+import org.apache.flink.util.AbstractID;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.function.Supplier;
+
+/** Default implementation of {@link RescaleTimeline}. */
+public class DefaultRescaleTimeline implements RescaleTimeline {
+
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultRescaleTimeline.class);
+
+ private final Supplier<JobInformation> jobInformationGetter;
+
+ private RescaleIdInfo rescaleIdInfo;
+
+ @Nullable private Rescale currentRescale;
+
+ private final BoundedFIFOQueue<Rescale> rescaleHistory;
+
+ private final Map<TerminalState, Rescale> latestRescales;
+
+ private final RescalesSummary rescalesSummary;
+
+ public DefaultRescaleTimeline(
+ Supplier<JobInformation> jobInformationGetter, int maxHistorySize) {
+ this.jobInformationGetter = jobInformationGetter;
+ this.rescaleIdInfo = new RescaleIdInfo(new AbstractID(), 0L);
+ this.latestRescales = new ConcurrentHashMap<>(TerminalState.values().length);
+ this.rescaleHistory = new BoundedFIFOQueue<>(maxHistorySize);
+ this.rescalesSummary = new RescalesSummary(maxHistorySize);
+ }
+
+ @Nullable
+ @Override
+ public Rescale latestRescale(TerminalState terminalState) {
Review Comment:
thanks, updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on PR #27285:
URL: https://github.com/apache/flink/pull/27285#issuecomment-3800206533
Hi, @davidradl Could you help take a look at this pr if you had the free time ?
Any input is appreciated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2732844680
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
Review Comment:
good idea.
updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on PR #27285:
URL: https://github.com/apache/flink/pull/27285#issuecomment-3806246470
Hi, @davidradl Thanks for your comments.
I have updated some-lines related. PTAL~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2732894567
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newCurrentRescale(boolean newRescaleEpoch);
+
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>ture</code> if update successfully <code>false</code> else.
+ */
+ boolean updateCurrentRescale(RescaleUpdater rescaleUpdater);
Review Comment:
> I am curious why we do not have an exception for the failure.
As mentioned here https://github.com/apache/flink/pull/27285#discussion_r2732850393
>Is there a non-current rescale?
if there's a non-current rescale(isIdling == true), the result of performance will be false.
> I wonder if the uuid should be returned here.
We only need the update result due to the reason like here https://github.com/apache/flink/pull/27285#discussion_r2732850393
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal closed pull request #27285: [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations
URL: https://github.com/apache/flink/pull/27285
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774946670
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>true</code> if there are no rescales to perform. <code>false</code> else.
+ */
+ boolean isIdling();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #isIdling()} must be
+ * true when creating a new current rescale. When there's a rescale in transition phase, we must
+ * seal the current resale before creating a new rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>true</code> if create successfully <code>false</code> else.
+ */
+ boolean newRescale(boolean newRescaleEpoch);
+
+ /** Get the latest rescale for the specified terminal state. */
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale. It only makes sense to update a rescale when there is an ongoing
+ * rescale that is in the process of transition states.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>true</code> if update successfully <code>false</code> else.
+ */
+ boolean updateRescale(RescaleUpdater rescaleUpdater);
+
+ /** Rescale operation interface. */
+ interface RescaleUpdater {
+ void update(Rescale rescaleToUpdate);
+ }
+
+ /** Default implementation of {@link RescaleTimeline} without any actions. */
Review Comment:
the original intention is `A default class with no actual method implementations.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728209697
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescalesSummary.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.util.stats.StatsSummary;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.Serializable;
+
+/** Statistics summary of rescales. */
+public class RescalesSummary implements Serializable {
+ private static final long serialVersionUID = 1L;
+
+ private static final Logger LOG = LoggerFactory.getLogger(RescalesSummary.class);
+
+ private final StatsSummary allTerminatedSummary;
+
+ private final StatsSummary completedRescalesSummary;
+ private final StatsSummary ignoredRescalesSummary;
+ private final StatsSummary failedRescalesSummary;
+
+ // Total terminated count
+ private long totalRescalesCount = 0L;
+ private long completedRescalesCount = 0L;
+
+ private long ignoredRescalesCount = 0L;
+ private long failedRescalesCount = 0L;
+ private long inProgressRescalesCount = 0L;
+
+ public RescalesSummary(int maxHistorySize) {
+ this.allTerminatedSummary = new StatsSummary(maxHistorySize);
+ this.completedRescalesSummary = new StatsSummary(maxHistorySize);
+ this.ignoredRescalesSummary = new StatsSummary(maxHistorySize);
+ this.failedRescalesSummary = new StatsSummary(maxHistorySize);
+ }
+
+ /**
+ * Add a terminated rescale in. Note, The method could be called after calling {@link
+ * #addInProgress(Rescale)}.
+ *
+ * @param rescale the target terminated rescale.
+ */
+ public void addTerminated(Rescale rescale) {
+ if (!Rescale.isTerminated(rescale)) {
+ LOG.warn(
+ "Unexpected rescale: {}, which will be ignored when computing statistics.",
+ rescale);
+ return;
+ }
+
+ allTerminatedSummary.add(rescale.getDuration().toMillis());
+ inProgressRescalesCount = 0;
+
+ if (rescale.getTerminalState() == null) {
+ return;
+ }
+
+ switch (rescale.getTerminalState()) {
+ case FAILED:
+ failedRescalesSummary.add(rescale.getDuration().toMillis());
+ failedRescalesCount++;
Review Comment:
not: is the size of `failedRescalesSummary` always the same as `failedRescalesCount`. Maybe we can remove the count variables unless you think it is more efficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728147328
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
Review Comment:
nit: line -> time line
Please could you give a move detailed description of this interface. why and when it is used and how the timeline relates to the rescaling.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
Review Comment:
javadoc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728167504
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
Review Comment:
I was expecting the rescale uuid to be returned (or is this something not to be exposed?) and an exception if the rescale could not be created.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on PR #27285:
URL: https://github.com/apache/flink/pull/27285#issuecomment-3883570841
Hi, @XComp Would you mind helping take look if you had the free time ?
Many thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2732897676
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescalesSummary.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.util.stats.StatsSummary;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.Serializable;
+
+/** Statistics summary of rescales. */
+public class RescalesSummary implements Serializable {
+ private static final long serialVersionUID = 1L;
+
+ private static final Logger LOG = LoggerFactory.getLogger(RescalesSummary.class);
+
+ private final StatsSummary allTerminatedSummary;
+
+ private final StatsSummary completedRescalesSummary;
+ private final StatsSummary ignoredRescalesSummary;
+ private final StatsSummary failedRescalesSummary;
+
+ // Total terminated count
+ private long totalRescalesCount = 0L;
+ private long completedRescalesCount = 0L;
+
+ private long ignoredRescalesCount = 0L;
+ private long failedRescalesCount = 0L;
+ private long inProgressRescalesCount = 0L;
+
+ public RescalesSummary(int maxHistorySize) {
+ this.allTerminatedSummary = new StatsSummary(maxHistorySize);
+ this.completedRescalesSummary = new StatsSummary(maxHistorySize);
+ this.ignoredRescalesSummary = new StatsSummary(maxHistorySize);
+ this.failedRescalesSummary = new StatsSummary(maxHistorySize);
+ }
+
+ /**
+ * Add a terminated rescale in. Note, The method could be called after calling {@link
+ * #addInProgress(Rescale)}.
+ *
+ * @param rescale the target terminated rescale.
+ */
+ public void addTerminated(Rescale rescale) {
+ if (!Rescale.isTerminated(rescale)) {
+ LOG.warn(
+ "Unexpected rescale: {}, which will be ignored when computing statistics.",
+ rescale);
+ return;
+ }
+
+ allTerminatedSummary.add(rescale.getDuration().toMillis());
+ inProgressRescalesCount = 0;
+
+ if (rescale.getTerminalState() == null) {
+ return;
+ }
+
+ switch (rescale.getTerminalState()) {
+ case FAILED:
+ failedRescalesSummary.add(rescale.getDuration().toMillis());
+ failedRescalesCount++;
Review Comment:
A nice proposal.
Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2732894567
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newCurrentRescale(boolean newRescaleEpoch);
+
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>ture</code> if update successfully <code>false</code> else.
+ */
+ boolean updateCurrentRescale(RescaleUpdater rescaleUpdater);
Review Comment:
> I am curious why we do not have an exception for the failure.
As mentioned here https://github.com/apache/flink/pull/27285#discussion_r2732850393
>Is there a non-current rescale?
if there's a non-current rescale(isIdling == true), the result of performance will be false.
> I wonder if the uuid should be returned here.
We only need the update result due to the reason like here https://github.com/apache/flink/pull/27285#discussion_r2732850393
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774808966
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>ture</code> if there are no rescales to perform. <code>false</code> else.
+ */
+ boolean isIdling();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #isIdling()} must be
+ * true when creating a new current rescale. When there's a rescale in transition phase, we must
+ * seal the current resale before creating a new rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newRescale(boolean newRescaleEpoch);
+
+ /** Get the latest rescale for the specified terminal state. */
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale. It only makes sense to update a rescale when there is an ongoing
+ * rescale that is in the process of transition states.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>ture</code> if update successfully <code>false</code> else.
Review Comment:
ture ->true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774869581
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>ture</code> if there are no rescales to perform. <code>false</code> else.
Review Comment:
@davidradl Nice catch! Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774929352
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>true</code> if there are no rescales to perform. <code>false</code> else.
+ */
+ boolean isIdling();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #isIdling()} must be
+ * true when creating a new current rescale. When there's a rescale in transition phase, we must
+ * seal the current resale before creating a new rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>true</code> if create successfully <code>false</code> else.
+ */
+ boolean newRescale(boolean newRescaleEpoch);
+
+ /** Get the latest rescale for the specified terminal state. */
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale. It only makes sense to update a rescale when there is an ongoing
+ * rescale that is in the process of transition states.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>true</code> if update successfully <code>false</code> else.
+ */
+ boolean updateRescale(RescaleUpdater rescaleUpdater);
+
+ /** Rescale operation interface. */
+ interface RescaleUpdater {
+ void update(Rescale rescaleToUpdate);
+ }
+
+ /** Default implementation of {@link RescaleTimeline} without any actions. */
Review Comment:
what does actions mean here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on PR #27285:
URL: https://github.com/apache/flink/pull/27285#issuecomment-3800392014
> Hi, @davidradl Could you help take a look at this pr if you had the free time ? Any input is appreciated!
@RocMarshal I have made some comments. Let me know what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728187075
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newCurrentRescale(boolean newRescaleEpoch);
+
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
Review Comment:
javadoc required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728157567
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
Review Comment:
the comments do not match the name of the method. If idling is a concept that is important to this interface then it should be described in the interface level javadoc
I do not understand what the `<code>ture</code> if contain <code>false</code> else.` means.
I wonder if `isIdling` is better than `inIdling`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774806817
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>ture</code> if there are no rescales to perform. <code>false</code> else.
+ */
+ boolean isIdling();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #isIdling()} must be
+ * true when creating a new current rescale. When there's a rescale in transition phase, we must
+ * seal the current resale before creating a new rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
+ */
+ boolean newRescale(boolean newRescaleEpoch);
+
+ /** Get the latest rescale for the specified terminal state. */
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale. It only makes sense to update a rescale when there is an ongoing
Review Comment:
If It only makes sense to update a rescale when there is an ongoing rescale, can there by a default method that does this checking and then calls the implementable method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774810104
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>ture</code> if there are no rescales to perform. <code>false</code> else.
+ */
+ boolean isIdling();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #isIdling()} must be
+ * true when creating a new current rescale. When there's a rescale in transition phase, we must
+ * seal the current resale before creating a new rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
Review Comment:
ture -> true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774896942
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
Review Comment:
My thoughts around this are:
Normally uuids are exposed as they are random tokens without inherent meaning that are returned on external interfaces.
Normally a create would return a uuid. If the caller needs to use this uuid to interact with it in the future.
Normally getters that return a boolean should start with is... as per the usual convention.
Is this a getter or a create?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774932121
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
+ */
+public interface RescaleTimeline {
+
+ /** Get the job information of current job. */
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or null, which represent no
+ * rescales in the current phase.
+ *
+ * @return <code>true</code> if there are no rescales to perform. <code>false</code> else.
+ */
+ boolean isIdling();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #isIdling()} must be
+ * true when creating a new current rescale. When there's a rescale in transition phase, we must
+ * seal the current resale before creating a new rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>true</code> if create successfully <code>false</code> else.
+ */
+ boolean newRescale(boolean newRescaleEpoch);
+
+ /** Get the latest rescale for the specified terminal state. */
+ @Nullable
+ Rescale latestRescale(TerminalState terminalState);
+
+ /**
+ * Update the current rescale. It only makes sense to update a rescale when there is an ongoing
+ * rescale that is in the process of transition states.
+ *
+ * @param rescaleUpdater The action to update the current rescale.
+ * @return <code>true</code> if update successfully <code>false</code> else.
+ */
+ boolean updateRescale(RescaleUpdater rescaleUpdater);
Review Comment:
The rescaleToUpdate must be from RescaleTimeline (the latest rescale in RescaleTimeline).
So, getting the latest rescale in RescaleTimeline maybe more convenient than getting it externally ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #27285:
URL: https://github.com/apache/flink/pull/27285#issuecomment-3589795214
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d3cc277a9eea4b57f661979e792d531c0998c4bd",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d3cc277a9eea4b57f661979e792d531c0998c4bd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d3cc277a9eea4b57f661979e792d531c0998c4bd UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728214403
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescalesSummary.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.util.stats.StatsSummary;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.Serializable;
+
+/** Statistics summary of rescales. */
+public class RescalesSummary implements Serializable {
+ private static final long serialVersionUID = 1L;
+
+ private static final Logger LOG = LoggerFactory.getLogger(RescalesSummary.class);
+
+ private final StatsSummary allTerminatedSummary;
+
+ private final StatsSummary completedRescalesSummary;
+ private final StatsSummary ignoredRescalesSummary;
+ private final StatsSummary failedRescalesSummary;
+
+ // Total terminated count
+ private long totalRescalesCount = 0L;
+ private long completedRescalesCount = 0L;
+
+ private long ignoredRescalesCount = 0L;
+ private long failedRescalesCount = 0L;
+ private long inProgressRescalesCount = 0L;
+
+ public RescalesSummary(int maxHistorySize) {
+ this.allTerminatedSummary = new StatsSummary(maxHistorySize);
+ this.completedRescalesSummary = new StatsSummary(maxHistorySize);
+ this.ignoredRescalesSummary = new StatsSummary(maxHistorySize);
+ this.failedRescalesSummary = new StatsSummary(maxHistorySize);
+ }
+
+ /**
+ * Add a terminated rescale in. Note, The method could be called after calling {@link
+ * #addInProgress(Rescale)}.
+ *
+ * @param rescale the target terminated rescale.
+ */
+ public void addTerminated(Rescale rescale) {
+ if (!Rescale.isTerminated(rescale)) {
+ LOG.warn(
+ "Unexpected rescale: {}, which will be ignored when computing statistics.",
+ rescale);
+ return;
+ }
+
+ allTerminatedSummary.add(rescale.getDuration().toMillis());
+ inProgressRescalesCount = 0;
Review Comment:
nit: `this.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2728167504
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/** The rescale line information updating interface. */
+public interface RescaleTimeline {
+
+ @Nullable
+ JobInformation getJobInformation();
+
+ /**
+ * Judge whether the current rescale in timeline is terminated or empty.
+ *
+ * @return <code>ture</code> if contain <code>false</code> else.
+ */
+ boolean inIdling();
+
+ boolean inRescalingProgress();
+
+ /**
+ * Create a new rescale and assign it as current rescale. Note, the {@link #inIdling()} must be
+ * idling when creating a new current rescale.
+ *
+ * @param newRescaleEpoch It represents whether the rescale resource requirements is in the new
+ * epoch.
+ * @return <code>ture</code> if create successfully <code>false</code> else.
Review Comment:
I was expecting the rescale uuid to be returned (or is this something not to be exposed?) and an exception if the rescale could not be created.
I suggest rewording the comment and renaming the variable . I am not understanding what is intended from the current comments and what would motivate me to put true or false here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774923080
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/DefaultRescaleTimeline.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+import org.apache.flink.runtime.util.BoundedFIFOQueue;
+import org.apache.flink.util.AbstractID;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.function.Supplier;
+
+/** Default implementation of {@link RescaleTimeline}. */
+public class DefaultRescaleTimeline implements RescaleTimeline {
+
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultRescaleTimeline.class);
+
+ private final Supplier<JobInformation> jobInformationGetter;
+
+ private RescaleIdInfo rescaleIdInfo;
+
+ @Nullable private Rescale currentRescale;
+
+ private final BoundedFIFOQueue<Rescale> rescaleHistory;
+
+ private final Map<TerminalState, Rescale> latestRescales;
+
+ private final RescalesSummary rescalesSummary;
+
+ public DefaultRescaleTimeline(
+ Supplier<JobInformation> jobInformationGetter, int maxHistorySize) {
+ this.jobInformationGetter = jobInformationGetter;
+ this.rescaleIdInfo = new RescaleIdInfo(new AbstractID(), 0L);
+ this.latestRescales = new ConcurrentHashMap<>(TerminalState.values().length);
+ this.rescaleHistory = new BoundedFIFOQueue<>(maxHistorySize);
+ this.rescalesSummary = new RescalesSummary(maxHistorySize);
+ }
+
+ @Nullable
+ @Override
+ public Rescale latestRescale(TerminalState terminalState) {
Review Comment:
suggest this is called getLatestRescale - so it looks like a getter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #27285:
URL: https://github.com/apache/flink/pull/27285#discussion_r2774958235
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/timeline/RescaleTimeline.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.adaptive.timeline;
+
+import org.apache.flink.runtime.scheduler.adaptive.allocator.JobInformation;
+
+import javax.annotation.Nullable;
+
+/**
+ * The rescale timeline information updating interface. When rescale history is enabled for a job,
+ * this class is used to perform fast operations on rescaling and to keep historical records. That
+ * is, a container used to record changes in rescaling.
Review Comment:
Yes, your comment is reasonable.
Updated it to `timeline`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38340][runtime] Introduce the rescale timeline related abstraction for performing rescale operations [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on PR #27285:
URL: https://github.com/apache/flink/pull/27285#issuecomment-3879024340
Hi, @ferenc-csaky Could you help take a look this patch ?
Any input is appreciated~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org