You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by ifndef-SleePy <gi...@git.apache.org> on 2018/01/10 06:47:08 UTC
[GitHub] flink pull request #5270: [FLINK-8368] [REST] Migrate SubtaskExecutionAttemp...
GitHub user ifndef-SleePy opened a pull request:
https://github.com/apache/flink/pull/5270
[FLINK-8368] [REST] Migrate SubtaskExecutionAttemptDetailsHandler to new a REST handler
## What is the purpose of the change
* Migrate `org.apache.flink.runtime.rest.handler.legacy.SubtaskExecutionAttemptDetailsHandler` to flip-6 `WebMonitorEndpoint`.
## Brief change log
* Make some abstraction about `JobVertexHandler` and `SubtaskAttemptHandler`.
* Add `SubtaskExecutionAttemptDetailsHandler` in flip-6 REST framework.
* Rename inner class `JobVertexMetrics` to public class `IOMetricsInfo`, make it more reusable.
## Verifying this change
* This change added unit tests.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/alibaba/flink FLINK-8368
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5270.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5270
----
commit 29182f04255f77288e207aef9e7015862d3e9a8c
Author: biao.liub <bi...@...>
Date: 2018-01-10T06:25:07Z
[FLINK-8368] Migrate org.apache.flink.runtime.rest.handler.legacy.SubtaskExecutionAttemptDetailsHandler to new a REST handler that registered in WebMonitorEndpoint
----
---
[GitHub] flink pull request #5270: [FLINK-8368] [REST] Migrate SubtaskExecutionAttemp...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5270
---
[GitHub] flink pull request #5270: [FLINK-8368] [REST] Migrate SubtaskExecutionAttemp...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5270#discussion_r161374482
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/job/JobVertexAccumulatorsHandler.java ---
@@ -65,25 +60,21 @@ public JobVertexAccumulatorsHandler(
}
@Override
- protected JobVertexAccumulatorsInfo handleRequest(HandlerRequest<EmptyRequestBody, JobVertexMessageParameters> request, AccessExecutionGraph executionGraph) throws RestHandlerException {
- JobVertexID jobVertexID = request.getPathParameter(JobVertexIdPathParameter.class);
- AccessExecutionJobVertex jobVertex = executionGraph.getJobVertex(jobVertexID);
-
- if (null != jobVertex) {
- StringifiedAccumulatorResult[] accs = jobVertex.getAggregatedUserAccumulatorsStringified();
- ArrayList<JobVertexAccumulatorsInfo.UserAccumulator> userAccumulatorList = new ArrayList<>(accs.length);
+ protected JobVertexAccumulatorsInfo handleRequest(
+ HandlerRequest<EmptyRequestBody, JobVertexMessageParameters> request,
+ AccessExecutionJobVertex jobVertex) throws RestHandlerException {
- for (StringifiedAccumulatorResult acc : accs) {
- userAccumulatorList.add(
- new JobVertexAccumulatorsInfo.UserAccumulator(
- acc.getName(),
- acc.getType(),
- acc.getValue()));
- }
+ StringifiedAccumulatorResult[] accs = jobVertex.getAggregatedUserAccumulatorsStringified();
+ ArrayList<JobVertexAccumulatorsInfo.UserAccumulator> userAccumulatorList = new ArrayList<>(accs.length);
- return new JobVertexAccumulatorsInfo(jobVertex.getJobVertexId().toString(), userAccumulatorList);
- } else {
- throw new RestHandlerException("There is no accumulator for vertex " + jobVertexID + '.', HttpResponseStatus.NOT_FOUND);
+ for (StringifiedAccumulatorResult acc : accs) {
+ userAccumulatorList.add(
+ new JobVertexAccumulatorsInfo.UserAccumulator(
+ acc.getName(),
+ acc.getType(),
+ acc.getValue()));
}
+
+ return new JobVertexAccumulatorsInfo(jobVertex.getJobVertexId().toString(), userAccumulatorList);
--- End diff --
Very nice refinement :-)
---
[GitHub] flink pull request #5270: [FLINK-8368] [REST] Migrate SubtaskExecutionAttemp...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5270#discussion_r161374497
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/job/SubtaskExecutionAttemptDetailsHandler.java ---
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.rest.handler.job;
+
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.api.common.time.Time;
+import org.apache.flink.runtime.execution.ExecutionState;
+import org.apache.flink.runtime.executiongraph.AccessExecution;
+import org.apache.flink.runtime.jobgraph.JobVertexID;
+import org.apache.flink.runtime.rest.handler.HandlerRequest;
+import org.apache.flink.runtime.rest.handler.RestHandlerException;
+import org.apache.flink.runtime.rest.handler.legacy.ExecutionGraphCache;
+import org.apache.flink.runtime.rest.handler.legacy.metrics.MetricFetcher;
+import org.apache.flink.runtime.rest.handler.util.MutableIOMetrics;
+import org.apache.flink.runtime.rest.messages.EmptyRequestBody;
+import org.apache.flink.runtime.rest.messages.JobIDPathParameter;
+import org.apache.flink.runtime.rest.messages.JobVertexIdPathParameter;
+import org.apache.flink.runtime.rest.messages.MessageHeaders;
+import org.apache.flink.runtime.rest.messages.job.SubtaskAttemptMessageParameters;
+import org.apache.flink.runtime.rest.messages.job.SubtaskExecutionAttemptDetailsInfo;
+import org.apache.flink.runtime.rest.messages.job.metrics.IOMetricsInfo;
+import org.apache.flink.runtime.taskmanager.TaskManagerLocation;
+import org.apache.flink.runtime.webmonitor.RestfulGateway;
+import org.apache.flink.runtime.webmonitor.retriever.GatewayRetriever;
+import org.apache.flink.util.Preconditions;
+
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.Executor;
+
+/**
+ * Handler of specific sub task execution attempt.
+ */
+public class SubtaskExecutionAttemptDetailsHandler extends AbstractSubtaskAttemptHandler<SubtaskExecutionAttemptDetailsInfo, SubtaskAttemptMessageParameters> {
+
+ private final MetricFetcher<?> metricFetcher;
+
+ /**
+ * Instantiates a new Abstract job vertex handler.
--- End diff --
wrong java docs: Should be `SubtaskExecutionAttemptDetailsHandler`.
---