Re: [PR] [api][java][python] Record embedding token usage metrics [flink-agents]

["wenjin272 (via GitHub)" <gi...@apache.org>]

wenjin272 commented on code in PR #1047:
URL: https://github.com/apache/flink-agents/pull/1047#discussion_r3912075877


##########
api/src/main/java/org/apache/flink/agents/api/embedding/model/BaseEmbeddingModelSetup.java:
##########
@@ -93,9 +94,52 @@ public String getModel() {
         return model;
     }
 
+    /**
+     * Record embedding token usage metrics for the given model on this setup's bound metric group.
+     *
+     * <p>Mirrors {@code BaseChatModelSetup#recordTokenMetrics} but records input-side tokens only,
+     * since embeddings have no completion tokens. Counters are placed under the same {@code model}
+     * key-value group used by chat metrics, so embedding and chat usage for a model share one
+     * dimension.
+     *
+     * <p>Unlike the chat path, embedding calls do not run inside a plan action that hands in a
+     * request-scoped metric group (vector-store, RAG, and direct calls reach this setup directly),
+     * so the resource-bound metric group injected via {@link #setMetricGroup} is used instead.
+     *
+     * @param modelName the name of the model used
+     * @param promptTokens the number of prompt tokens
+     * @param totalTokens the total number of tokens reported by the provider
+     */
+    public void recordTokenMetrics(String modelName, long promptTokens, long totalTokens) {
+        Preconditions.checkArgument(
+                modelName != null && !modelName.isBlank(), "Model name must not be null or blank.");
+        FlinkAgentsMetricGroup metricGroup = getMetricGroup();

Review Comment:
   For context, this is the same recording-boundary issue that led #870 to remove automatic metric recording and retain only usage transport. @joeyutong  is working on the framework-level fix, so this PR may need to wait for that work and then adopt the resulting API.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org