[GitHub] [pulsar] jerrypeng commented on a change in pull request #7641: Export Function worker internal stats via Prometheus

[GitBox <gi...@apache.org>]

jerrypeng commented on a change in pull request #7641:
URL: https://github.com/apache/pulsar/pull/7641#discussion_r459705497



##########
File path: pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/WorkerStatsManager.java
##########
@@ -0,0 +1,250 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pulsar.functions.worker;
+
+import io.prometheus.client.CollectorRegistry;
+import io.prometheus.client.Gauge;
+import io.prometheus.client.Summary;
+import lombok.Setter;
+import org.apache.pulsar.functions.instance.stats.PrometheusTextFormat;
+
+import java.io.IOException;
+import java.io.StringWriter;
+
+public class WorkerStatsManager {
+
+  private static final String PULSAR_FUNCTION_WORKER_METRICS_PREFIX = "pulsar_function_worker_";
+  private static final String START_UP_TIME = "start_up_time_ms";
+  private static final String INSTANCE_COUNT = "instance_count";
+  private static final String SCHEDULE_TOTAL_EXEC_TIME = "schedule_execution_time_total_ms";
+  private static final String SCHEDULE_STRATEGY_EXEC_TIME = "schedule_strategy_execution_time_ms";
+  private static final String REBALANCE_TOTAL_EXEC_TIME = "rebalance_execution_time_total_ms";
+  private static final String REBALANCE_STRATEGY_EXEC_TIME = "rebalance_strategy_execution_time_ms";
+  private static final String STOPPING_INSTANCE_PROCESS_TIME = "stop_instance_process_time_ms";
+  private static final String UPDATING_INSTANCE_PROCESS_TIME = "update_instance_process_time_ms";
+  private static final String STARTING_INSTANCE_PROCESS_TIME = "start_instance_process_time_ms";

Review comment:
       > Do we also want to measure number of function instances started/stopped/terminated.
   A count is already include e.g. 
   
   pulsar_function_worker_stop_instance_process_time_ms_count 10
   
   > What about differentiating between sources/sinks/functions?
   
   Let me think more on that.  The thing is this is more of a backend metric (Assignment processing workflow).  We don't really differentiate between sources/sinks/functions at that level.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org