You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by ifndef-SleePy <gi...@git.apache.org> on 2018/01/10 06:47:08 UTC

[GitHub] flink pull request #5270: [FLINK-8368] [REST] Migrate SubtaskExecutionAttemp...

GitHub user ifndef-SleePy opened a pull request:

    https://github.com/apache/flink/pull/5270

    [FLINK-8368] [REST] Migrate SubtaskExecutionAttemptDetailsHandler to new a REST handler

    ## What is the purpose of the change
    
    * Migrate `org.apache.flink.runtime.rest.handler.legacy.SubtaskExecutionAttemptDetailsHandler` to flip-6 `WebMonitorEndpoint`.
    
    ## Brief change log
    
    * Make some abstraction about `JobVertexHandler` and `SubtaskAttemptHandler`.
    * Add `SubtaskExecutionAttemptDetailsHandler` in flip-6 REST framework.
    * Rename inner class `JobVertexMetrics` to public class `IOMetricsInfo`, make it more reusable.
    
    ## Verifying this change
    
    * This change added unit tests.
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): (no)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
      - The serializers: (no)
      - The runtime per-record code paths (performance sensitive): (no)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
      - The S3 file system connector: (no)
    
    ## Documentation
    
      - Does this pull request introduce a new feature? (no)
      - If yes, how is the feature documented? (not applicable)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/alibaba/flink FLINK-8368

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5270.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5270
    
----
commit 29182f04255f77288e207aef9e7015862d3e9a8c
Author: biao.liub <bi...@...>
Date:   2018-01-10T06:25:07Z

    [FLINK-8368] Migrate org.apache.flink.runtime.rest.handler.legacy.SubtaskExecutionAttemptDetailsHandler to new a REST handler that registered in WebMonitorEndpoint

----


---

[GitHub] flink pull request #5270: [FLINK-8368] [REST] Migrate SubtaskExecutionAttemp...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/flink/pull/5270


---

[GitHub] flink pull request #5270: [FLINK-8368] [REST] Migrate SubtaskExecutionAttemp...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5270#discussion_r161374482
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/job/JobVertexAccumulatorsHandler.java ---
    @@ -65,25 +60,21 @@ public JobVertexAccumulatorsHandler(
     	}
     
     	@Override
    -	protected JobVertexAccumulatorsInfo handleRequest(HandlerRequest<EmptyRequestBody, JobVertexMessageParameters> request, AccessExecutionGraph executionGraph) throws RestHandlerException {
    -		JobVertexID jobVertexID = request.getPathParameter(JobVertexIdPathParameter.class);
    -		AccessExecutionJobVertex jobVertex = executionGraph.getJobVertex(jobVertexID);
    -
    -		if (null != jobVertex) {
    -			StringifiedAccumulatorResult[] accs = jobVertex.getAggregatedUserAccumulatorsStringified();
    -			ArrayList<JobVertexAccumulatorsInfo.UserAccumulator> userAccumulatorList = new ArrayList<>(accs.length);
    +	protected JobVertexAccumulatorsInfo handleRequest(
    +			HandlerRequest<EmptyRequestBody, JobVertexMessageParameters> request,
    +			AccessExecutionJobVertex jobVertex) throws RestHandlerException {
     
    -			for (StringifiedAccumulatorResult acc : accs) {
    -				userAccumulatorList.add(
    -					new JobVertexAccumulatorsInfo.UserAccumulator(
    -						acc.getName(),
    -						acc.getType(),
    -						acc.getValue()));
    -			}
    +		StringifiedAccumulatorResult[] accs = jobVertex.getAggregatedUserAccumulatorsStringified();
    +		ArrayList<JobVertexAccumulatorsInfo.UserAccumulator> userAccumulatorList = new ArrayList<>(accs.length);
     
    -			return new JobVertexAccumulatorsInfo(jobVertex.getJobVertexId().toString(), userAccumulatorList);
    -		} else {
    -			throw new RestHandlerException("There is no accumulator for vertex " + jobVertexID + '.', HttpResponseStatus.NOT_FOUND);
    +		for (StringifiedAccumulatorResult acc : accs) {
    +			userAccumulatorList.add(
    +				new JobVertexAccumulatorsInfo.UserAccumulator(
    +					acc.getName(),
    +					acc.getType(),
    +					acc.getValue()));
     		}
    +
    +		return new JobVertexAccumulatorsInfo(jobVertex.getJobVertexId().toString(), userAccumulatorList);
    --- End diff --
    
    Very nice refinement :-)


---

[GitHub] flink pull request #5270: [FLINK-8368] [REST] Migrate SubtaskExecutionAttemp...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5270#discussion_r161374497
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/job/SubtaskExecutionAttemptDetailsHandler.java ---
    @@ -0,0 +1,131 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.flink.runtime.rest.handler.job;
    +
    +import org.apache.flink.api.common.JobID;
    +import org.apache.flink.api.common.time.Time;
    +import org.apache.flink.runtime.execution.ExecutionState;
    +import org.apache.flink.runtime.executiongraph.AccessExecution;
    +import org.apache.flink.runtime.jobgraph.JobVertexID;
    +import org.apache.flink.runtime.rest.handler.HandlerRequest;
    +import org.apache.flink.runtime.rest.handler.RestHandlerException;
    +import org.apache.flink.runtime.rest.handler.legacy.ExecutionGraphCache;
    +import org.apache.flink.runtime.rest.handler.legacy.metrics.MetricFetcher;
    +import org.apache.flink.runtime.rest.handler.util.MutableIOMetrics;
    +import org.apache.flink.runtime.rest.messages.EmptyRequestBody;
    +import org.apache.flink.runtime.rest.messages.JobIDPathParameter;
    +import org.apache.flink.runtime.rest.messages.JobVertexIdPathParameter;
    +import org.apache.flink.runtime.rest.messages.MessageHeaders;
    +import org.apache.flink.runtime.rest.messages.job.SubtaskAttemptMessageParameters;
    +import org.apache.flink.runtime.rest.messages.job.SubtaskExecutionAttemptDetailsInfo;
    +import org.apache.flink.runtime.rest.messages.job.metrics.IOMetricsInfo;
    +import org.apache.flink.runtime.taskmanager.TaskManagerLocation;
    +import org.apache.flink.runtime.webmonitor.RestfulGateway;
    +import org.apache.flink.runtime.webmonitor.retriever.GatewayRetriever;
    +import org.apache.flink.util.Preconditions;
    +
    +import java.util.Map;
    +import java.util.concurrent.CompletableFuture;
    +import java.util.concurrent.Executor;
    +
    +/**
    + * Handler of specific sub task execution attempt.
    + */
    +public class SubtaskExecutionAttemptDetailsHandler extends AbstractSubtaskAttemptHandler<SubtaskExecutionAttemptDetailsInfo, SubtaskAttemptMessageParameters> {
    +
    +	private final MetricFetcher<?> metricFetcher;
    +
    +	/**
    +	 * Instantiates a new Abstract job vertex handler.
    --- End diff --
    
    wrong java docs: Should be `SubtaskExecutionAttemptDetailsHandler`.


---