You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "tsaiggo (via GitHub)" <gi...@apache.org> on 2025/10/29 13:25:50 UTC
[PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
tsaiggo opened a new pull request, #290:
URL: https://github.com/apache/flink-agents/pull/290
<!--
* Thank you very much for contributing to Flink Agents.
* Please add the relevant components in the PR title. E.g., [api], [runtime], [java], [python], [hotfix], etc.
-->
<!-- Please link the PR to the relevant issue(s). Hotfix doesn't need this. -->
Linked issue: https://github.com/apache/flink-agents/issues/286
### Purpose of change
<!-- What is the purpose of this change? -->
This PR adds support for the Azure AI chat model as a foundational model for the agent.
### Tests
<!-- How is this change verified? -->
I added one test to verify this change, but I ran into some issues while running the test cases:
1. When I tried to run the `AgentWithOllamaExample`, the output stream didn't produce any data. Consequently, the same thing happened when I tested my new `AgentWithAzureAIExample`. My assumption is that this isn't related to the chat model implementation itself, but is more likely a higher-level issue (perhaps in the core agent logic or streaming setup).
2. Because of this, I switched to a different test case, `ReActAgentExample`. I modified it locally to use my Azure AI implementation, and it worked perfectly. However, I did not commit this change, as my PR should not introduce breaking changes to existing examples.
3. Finally, since this PR integrates a cloud-hosted model, I cannot check in my personal endpoint or API key. Therefore, I created the new `AgentWithAzureAIExample` and leveraged the `MockChatModel` (from `AgentWithResource`) to add mock tests. The "happy path" for this mock test passes successfully.
### API
<!-- Does this change touches any public APIs? -->
No
### Documentation
<!-- Should this change be covered by the user documentation?-->
Yes, exactly. It's because we now offer more model options.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2492524895
##########
examples/pom.xml:
##########
@@ -64,6 +64,11 @@ under the License.
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-agents-integrations-chat-models-azureai</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
Yes, I think the dependency should be introduced only when it's needed. Thanks for addressing my comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2490238355
##########
examples/pom.xml:
##########
@@ -64,6 +64,11 @@ under the License.
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-agents-integrations-chat-models-azureai</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
Why do we need this dependency in examples/ ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "tsaiggo (via GitHub)" <gi...@apache.org>.
tsaiggo commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2486654925
##########
integrations/chat-models/azureai/src/main/java/org/apache/flink/agents/integrations/chatmodels/azureai/AzureAIChatModelSetup.java:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.agents.integrations.chatmodels.azureai;
+
+import org.apache.flink.agents.api.chat.model.BaseChatModelSetup;
+import org.apache.flink.agents.api.resource.Resource;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.apache.flink.agents.api.resource.ResourceType;
+
+/**
+ * A chat model integration for Azure AI Chat Completions service.
+ *
+ * <p>This implementation adapts the generic Flink Agents chat model interface to the Azure AI Chat
+ * Completions API.
+ *
+ * <p>See also {@link BaseChatModelSetup} for the common resource abstractions and lifecycle.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * public class MyAgent extends Agent {
+ * // Register the chat model setup via @ChatModelSetup metadata.
+ * @ChatModelSetup
+ * public static ResourceDesc azureAI() {
+ * return ResourceDescriptor.Builder.newBuilder(AzureAIChatModelSetup.class.getName())
+ * .addInitialArgument("model", "<your-azure-ai-model-name>")
+ * .addInitialArgument("prompt", "<your-prompt-template>")
+ * .addInitialArgument("tools", "<your-tool-list>")
+ * .addInitialArgument("key", "<your-azure-ai-key>")
+ * .addInitialArgument("endpoint", "<your-azure-ai-endpoint>")
+ * .build();
+ * }
+ * }
+ * }</pre>
+ */
+public class AzureAIChatModelSetup extends BaseChatModelSetup {
+
+ public AzureAIChatModelSetup(
+ ResourceDescriptor descriptor,
+ java.util.function.BiFunction<String, ResourceType, Resource> getResource) {
+ super(descriptor, getResource);
+ }
+
+ // For any other specific parameters, please refer to ChatCompletionsOptions @Override
Review Comment:
resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "tsaiggo (via GitHub)" <gi...@apache.org>.
tsaiggo commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2490499509
##########
examples/pom.xml:
##########
@@ -64,6 +64,11 @@ under the License.
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-agents-integrations-chat-models-azureai</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
I prefer to keep documentation updates in a separate PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2492525714
##########
examples/pom.xml:
##########
@@ -64,6 +64,11 @@ under the License.
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-agents-integrations-chat-models-azureai</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
FYI, I just created #305 for the documentation work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "tsaiggo (via GitHub)" <gi...@apache.org>.
tsaiggo commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2485440251
##########
integrations/chat-models/azureai/src/main/java/org/apache/flink/agents/integrations/chatmodels/azureai/AzureAIChatModelSetup.java:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.agents.integrations.chatmodels.azureai;
+
+import org.apache.flink.agents.api.chat.model.BaseChatModelSetup;
+import org.apache.flink.agents.api.resource.Resource;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.apache.flink.agents.api.resource.ResourceType;
+
+/**
+ * A chat model integration for Azure AI Chat Completions service.
+ *
+ * <p>This implementation adapts the generic Flink Agents chat model interface to the Azure AI Chat
+ * Completions API.
+ *
+ * <p>See also {@link BaseChatModelSetup} for the common resource abstractions and lifecycle.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * public class MyAgent extends Agent {
+ * // Register the chat model setup via @ChatModelSetup metadata.
+ * @ChatModelSetup
+ * public static ResourceDesc azureAI() {
+ * return ResourceDescriptor.Builder.newBuilder(AzureAIChatModelSetup.class.getName())
+ * .addInitialArgument("model", "<your-azure-ai-model-name>")
+ * .addInitialArgument("prompt", "<your-prompt-template>")
+ * .addInitialArgument("tools", "<your-tool-list>")
+ * .addInitialArgument("key", "<your-azure-ai-key>")
+ * .addInitialArgument("endpoint", "<your-azure-ai-endpoint>")
+ * .build();
+ * }
+ * }
+ * }</pre>
+ */
+public class AzureAIChatModelSetup extends BaseChatModelSetup {
+
+ public AzureAIChatModelSetup(
+ ResourceDescriptor descriptor,
+ java.util.function.BiFunction<String, ResourceType, Resource> getResource) {
+ super(descriptor, getResource);
+ }
+
+ // For any other specific parameters, please refer to ChatCompletionsOptions @Override
Review Comment:
Oops, good catch! 😂 Blame it on the late-night brain fog. Will fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "tsaiggo (via GitHub)" <gi...@apache.org>.
tsaiggo commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2490668507
##########
examples/pom.xml:
##########
@@ -64,6 +64,11 @@ under the License.
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-agents-integrations-chat-models-azureai</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2480331186
##########
integrations/chat-models/azureai/src/main/java/org/apache/flink/agents/integrations/chatmodels/azureai/AzureAIChatModelSetup.java:
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.agents.integrations.chatmodels.azureai;
+
+import org.apache.flink.agents.api.chat.model.BaseChatModelSetup;
+import org.apache.flink.agents.api.resource.Resource;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.apache.flink.agents.api.resource.ResourceType;
+
+/**
+ * A chat model integration for Azure AI Chat Completions service.
+ *
+ * <p>This implementation adapts the generic Flink Agents chat model interface to the Azure AI Chat
+ * Completions API.
+ *
+ * <p>See also {@link BaseChatModelSetup} for the common resource abstractions and lifecycle.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * public class MyAgent extends Agent {
+ * // Register the chat model setup via @ChatModelSetup metadata.
+ * @ChatModelSetup
+ * public static ResourceDesc azureAI() {
+ * return ResourceDescriptor.Builder.newBuilder(AzureAIChatModelSetup.class.getName())
+ * .addInitialArgument("model", "<your-azure-ai-model-name>")
+ * .addInitialArgument("prompt", "<your-prompt-template>")
+ * .addInitialArgument("tools", "<your-tool-list>")
+ * .addInitialArgument("key", "<your-azure-ai-key>")
+ * .addInitialArgument("endpoint", "<your-azure-ai-endpoint>")
+ * .build();
+ * }
+ * }
+ * }</pre>
+ */
+public class AzureAIChatModelSetup extends BaseChatModelSetup {
+
+ private final String model;
+ private final String prompt;
+ private final java.util.List<String> tools;
+ private final String key;
+ private final String endpoint;
+
+ public AzureAIChatModelSetup(
+ ResourceDescriptor descriptor,
+ java.util.function.BiFunction<String, ResourceType, Resource> getResource) {
+ super(descriptor, getResource);
+ this.model = descriptor.getArgument("model");
+ this.prompt = descriptor.getArgument("prompt");
+ this.tools = (java.util.List<String>) descriptor.getArgument("tools");
+ this.key = descriptor.getArgument("key");
+ this.endpoint = descriptor.getArgument("endpoint");
Review Comment:
`key` and `endpoint` should be the arguments of `AzureAIChatModelConnection`. Different `AzureAIChatModelSetup` can share the same `AzureAIChatModelConnection`.
##########
integrations/chat-models/azureai/src/main/java/org/apache/flink/agents/integrations/chatmodels/azureai/AzureAIChatModelConnection.java:
##########
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.agents.integrations.chatmodels.azureai;
+
+import com.azure.ai.inference.ChatCompletionsClient;
+import com.azure.ai.inference.ChatCompletionsClientBuilder;
+import com.azure.ai.inference.models.*;
+import com.azure.core.credential.AzureKeyCredential;
+import com.azure.core.util.BinaryData;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.flink.agents.api.chat.messages.ChatMessage;
+import org.apache.flink.agents.api.chat.messages.MessageRole;
+import org.apache.flink.agents.api.chat.model.BaseChatModelConnection;
+import org.apache.flink.agents.api.resource.Resource;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.apache.flink.agents.api.resource.ResourceType;
+import org.apache.flink.agents.api.tools.Tool;
+
+import java.util.*;
+import java.util.function.BiFunction;
+import java.util.stream.Collectors;
+
+/**
+ * A chat model integration for Azure AI Chat Completions service.
+ *
+ * <p>This implementation adapts the generic Flink Agents chat model interface to the Azure AI Chat
+ * Completions API.
+ *
+ * <p>See also {@link BaseChatModelConnection} for the common resource abstractions and lifecycle.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * public class MyAgent extends Agent {
+ * // Register the chat model connection via @ChatModelConnection metadata.
+ * @ChatModelConnection
+ * public static ResourceDesc azureAI() {
+ * return ResourceDescriptor.Builder.newBuilder(AzureAIChatModelConnection.class.getName())
+ * .addInitialArgument("endpoint", "<your-azure-ai-endpoint>")
+ * .addInitialArgument("apiKey", "<your-azure-ai-api-key>")
+ * .build();
+ * }
+ * }
+ * }</pre>
+ */
+public class AzureAIChatModelConnection extends BaseChatModelConnection {
+
+ private final ChatCompletionsClient client;
+
+ /**
+ * Creates a new AzureAI chat model connection.
+ *
+ * @param descriptor a resource descriptor contains the initial parameters
+ * @param getResource a function to resolve resources (e.g., tools) by name and type
+ * @throws IllegalArgumentException if endpoint is null or empty
+ */
+ public AzureAIChatModelConnection(
+ ResourceDescriptor descriptor, BiFunction<String, ResourceType, Resource> getResource) {
+ super(descriptor, getResource);
+
+ String endpoint = descriptor.getArgument("endpoint");
+ String apiKey = descriptor.getArgument("apiKey");
+ if (endpoint == null || endpoint.isEmpty()) {
+ throw new IllegalArgumentException("endpoint should not be null or empty.");
+ }
+ this.client =
+ new ChatCompletionsClientBuilder()
+ .credential(new AzureKeyCredential(apiKey))
+ .endpoint(endpoint)
+ .buildClient();
+ }
+
+ @SuppressWarnings("unchecked")
Review Comment:
My idea show this is a redundant suppression.
##########
integrations/chat-models/azureai/src/main/java/org/apache/flink/agents/integrations/chatmodels/azureai/AzureAIChatModelSetup.java:
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.agents.integrations.chatmodels.azureai;
+
+import org.apache.flink.agents.api.chat.model.BaseChatModelSetup;
+import org.apache.flink.agents.api.resource.Resource;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.apache.flink.agents.api.resource.ResourceType;
+
+/**
+ * A chat model integration for Azure AI Chat Completions service.
+ *
+ * <p>This implementation adapts the generic Flink Agents chat model interface to the Azure AI Chat
+ * Completions API.
+ *
+ * <p>See also {@link BaseChatModelSetup} for the common resource abstractions and lifecycle.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * public class MyAgent extends Agent {
+ * // Register the chat model setup via @ChatModelSetup metadata.
+ * @ChatModelSetup
+ * public static ResourceDesc azureAI() {
+ * return ResourceDescriptor.Builder.newBuilder(AzureAIChatModelSetup.class.getName())
+ * .addInitialArgument("model", "<your-azure-ai-model-name>")
+ * .addInitialArgument("prompt", "<your-prompt-template>")
+ * .addInitialArgument("tools", "<your-tool-list>")
+ * .addInitialArgument("key", "<your-azure-ai-key>")
+ * .addInitialArgument("endpoint", "<your-azure-ai-endpoint>")
+ * .build();
+ * }
+ * }
+ * }</pre>
+ */
+public class AzureAIChatModelSetup extends BaseChatModelSetup {
+
+ private final String model;
+ private final String prompt;
+ private final java.util.List<String> tools;
+ private final String key;
+ private final String endpoint;
+
+ public AzureAIChatModelSetup(
+ ResourceDescriptor descriptor,
+ java.util.function.BiFunction<String, ResourceType, Resource> getResource) {
+ super(descriptor, getResource);
+ this.model = descriptor.getArgument("model");
+ this.prompt = descriptor.getArgument("prompt");
+ this.tools = (java.util.List<String>) descriptor.getArgument("tools");
Review Comment:
`prompt` and `tools` will be retrieved in constructor of base class `BaseChatModelSetup`, no need to process here again.
##########
integrations/chat-models/azureai/src/main/java/org/apache/flink/agents/integrations/chatmodels/azureai/AzureAIChatModelSetup.java:
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.agents.integrations.chatmodels.azureai;
+
+import org.apache.flink.agents.api.chat.model.BaseChatModelSetup;
+import org.apache.flink.agents.api.resource.Resource;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.apache.flink.agents.api.resource.ResourceType;
+
+/**
+ * A chat model integration for Azure AI Chat Completions service.
+ *
+ * <p>This implementation adapts the generic Flink Agents chat model interface to the Azure AI Chat
+ * Completions API.
+ *
+ * <p>See also {@link BaseChatModelSetup} for the common resource abstractions and lifecycle.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * public class MyAgent extends Agent {
+ * // Register the chat model setup via @ChatModelSetup metadata.
+ * @ChatModelSetup
+ * public static ResourceDesc azureAI() {
+ * return ResourceDescriptor.Builder.newBuilder(AzureAIChatModelSetup.class.getName())
+ * .addInitialArgument("model", "<your-azure-ai-model-name>")
+ * .addInitialArgument("prompt", "<your-prompt-template>")
+ * .addInitialArgument("tools", "<your-tool-list>")
+ * .addInitialArgument("key", "<your-azure-ai-key>")
+ * .addInitialArgument("endpoint", "<your-azure-ai-endpoint>")
+ * .build();
+ * }
+ * }
+ * }</pre>
+ */
+public class AzureAIChatModelSetup extends BaseChatModelSetup {
+
+ private final String model;
+ private final String prompt;
+ private final java.util.List<String> tools;
+ private final String key;
+ private final String endpoint;
+
+ public AzureAIChatModelSetup(
+ ResourceDescriptor descriptor,
+ java.util.function.BiFunction<String, ResourceType, Resource> getResource) {
+ super(descriptor, getResource);
+ this.model = descriptor.getArgument("model");
+ this.prompt = descriptor.getArgument("prompt");
+ this.tools = (java.util.List<String>) descriptor.getArgument("tools");
+ this.key = descriptor.getArgument("key");
+ this.endpoint = descriptor.getArgument("endpoint");
+ }
+
+ public AzureAIChatModelSetup(
+ String model,
+ String prompt,
+ java.util.List<String> tools,
+ String key,
+ String endpoint,
+ java.util.function.BiFunction<String, ResourceType, Resource> getResource) {
+ this(
+ new ResourceDescriptor(
+ AzureAIChatModelSetup.class.getName(),
+ java.util.Map.of(
+ "model", model,
+ "prompt", prompt,
+ "tools", tools,
+ "key", key,
+ "endpoint", endpoint)),
+ getResource);
+ }
+
+ @Override
+ public java.util.Map<String, Object> getParameters() {
+ java.util.Map<String, Object> params = new java.util.HashMap<>();
+ params.put("model", model);
+ params.put("prompt", prompt);
+ params.put("tools", tools);
Review Comment:
Base class `BaseChatModelSetup ` will process the `prompt` and `tools` directly, no need to pass them as parameters here.
##########
integrations/chat-models/azureai/src/main/java/org/apache/flink/agents/integrations/chatmodels/azureai/AzureAIChatModelConnection.java:
##########
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.agents.integrations.chatmodels.azureai;
+
+import com.azure.ai.inference.ChatCompletionsClient;
+import com.azure.ai.inference.ChatCompletionsClientBuilder;
+import com.azure.ai.inference.models.*;
+import com.azure.core.credential.AzureKeyCredential;
+import com.azure.core.util.BinaryData;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.flink.agents.api.chat.messages.ChatMessage;
+import org.apache.flink.agents.api.chat.messages.MessageRole;
+import org.apache.flink.agents.api.chat.model.BaseChatModelConnection;
+import org.apache.flink.agents.api.resource.Resource;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.apache.flink.agents.api.resource.ResourceType;
+import org.apache.flink.agents.api.tools.Tool;
+
+import java.util.*;
+import java.util.function.BiFunction;
+import java.util.stream.Collectors;
+
+/**
+ * A chat model integration for Azure AI Chat Completions service.
+ *
+ * <p>This implementation adapts the generic Flink Agents chat model interface to the Azure AI Chat
+ * Completions API.
+ *
+ * <p>See also {@link BaseChatModelConnection} for the common resource abstractions and lifecycle.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * public class MyAgent extends Agent {
+ * // Register the chat model connection via @ChatModelConnection metadata.
+ * @ChatModelConnection
+ * public static ResourceDesc azureAI() {
+ * return ResourceDescriptor.Builder.newBuilder(AzureAIChatModelConnection.class.getName())
+ * .addInitialArgument("endpoint", "<your-azure-ai-endpoint>")
+ * .addInitialArgument("apiKey", "<your-azure-ai-api-key>")
+ * .build();
+ * }
+ * }
+ * }</pre>
+ */
+public class AzureAIChatModelConnection extends BaseChatModelConnection {
+
+ private final ChatCompletionsClient client;
+
+ /**
+ * Creates a new AzureAI chat model connection.
+ *
+ * @param descriptor a resource descriptor contains the initial parameters
+ * @param getResource a function to resolve resources (e.g., tools) by name and type
+ * @throws IllegalArgumentException if endpoint is null or empty
+ */
+ public AzureAIChatModelConnection(
+ ResourceDescriptor descriptor, BiFunction<String, ResourceType, Resource> getResource) {
+ super(descriptor, getResource);
+
+ String endpoint = descriptor.getArgument("endpoint");
+ String apiKey = descriptor.getArgument("apiKey");
+ if (endpoint == null || endpoint.isEmpty()) {
+ throw new IllegalArgumentException("endpoint should not be null or empty.");
+ }
+ this.client =
+ new ChatCompletionsClientBuilder()
+ .credential(new AzureKeyCredential(apiKey))
+ .endpoint(endpoint)
+ .buildClient();
+ }
+
+ @SuppressWarnings("unchecked")
+ private List<ChatCompletionsToolDefinition> convertToAzureAITools(List<Tool> tools) {
+ final ObjectMapper mapper = new ObjectMapper();
+ final List<ChatCompletionsToolDefinition> azureAITools = new ArrayList<>();
+ try {
+ for (Tool tool : tools) {
+ final Map<String, Object> schema =
+ mapper.readValue(
+ tool.getMetadata().getInputSchema(), new TypeReference<>() {});
+
+ final FunctionDefinition functionDef =
+ new FunctionDefinition(tool.getName())
+ .setDescription(tool.getDescription())
+ .setParameters(BinaryData.fromObject(schema));
+
+ azureAITools.add(new ChatCompletionsFunctionToolDefinition(functionDef));
+ }
+ return azureAITools;
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ private ChatRequestMessage convertToChatRequestMessage(ChatMessage message) {
+ final String content = message.getContent();
+ final MessageRole role = message.getRole();
+ switch (role) {
+ case SYSTEM:
+ return new ChatRequestSystemMessage(content);
+ case USER:
+ return new ChatRequestUserMessage(content);
+ case ASSISTANT:
+ return new ChatRequestAssistantMessage(content);
+ case TOOL:
+ return new ChatRequestToolMessage(content);
Review Comment:
Here, the `content` will be set as `toolCallId` of `ChatRequestToolMessage`, so we should pass the function call id generated by Azure llm.
```
ChatRequestToolMessage msg =
new ChatRequestToolMessage(
(String) message.getExtraArgs().get("externalId"));
msg.setContent(content);
return msg;
```
and the `externalId` should be set in `convertToAgentsTools`:
```
private List<Map<String, Object>> convertToAgentsTools(
List<ChatCompletionsToolCall> azureToolCalls) {
...
final Map<String, Object> call =
Map.of(
"id", functionCall.getId(),
"original_id", functionCall.getId(),
"type", "function",
"function",
Map.of(
"name", functionCall.getFunction().getName(),
"arguments",
functionCall.getFunction().getArguments()));
toolCalls.add(call);
...
}
```
##########
integrations/chat-models/azureai/src/main/java/org/apache/flink/agents/integrations/chatmodels/azureai/AzureAIChatModelConnection.java:
##########
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.agents.integrations.chatmodels.azureai;
+
+import com.azure.ai.inference.ChatCompletionsClient;
+import com.azure.ai.inference.ChatCompletionsClientBuilder;
+import com.azure.ai.inference.models.*;
+import com.azure.core.credential.AzureKeyCredential;
+import com.azure.core.util.BinaryData;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.flink.agents.api.chat.messages.ChatMessage;
+import org.apache.flink.agents.api.chat.messages.MessageRole;
+import org.apache.flink.agents.api.chat.model.BaseChatModelConnection;
+import org.apache.flink.agents.api.resource.Resource;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.apache.flink.agents.api.resource.ResourceType;
+import org.apache.flink.agents.api.tools.Tool;
+
+import java.util.*;
+import java.util.function.BiFunction;
+import java.util.stream.Collectors;
+
+/**
+ * A chat model integration for Azure AI Chat Completions service.
+ *
+ * <p>This implementation adapts the generic Flink Agents chat model interface to the Azure AI Chat
+ * Completions API.
+ *
+ * <p>See also {@link BaseChatModelConnection} for the common resource abstractions and lifecycle.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * public class MyAgent extends Agent {
+ * // Register the chat model connection via @ChatModelConnection metadata.
+ * @ChatModelConnection
+ * public static ResourceDesc azureAI() {
+ * return ResourceDescriptor.Builder.newBuilder(AzureAIChatModelConnection.class.getName())
+ * .addInitialArgument("endpoint", "<your-azure-ai-endpoint>")
+ * .addInitialArgument("apiKey", "<your-azure-ai-api-key>")
+ * .build();
+ * }
+ * }
+ * }</pre>
+ */
+public class AzureAIChatModelConnection extends BaseChatModelConnection {
+
+ private final ChatCompletionsClient client;
+
+ /**
+ * Creates a new AzureAI chat model connection.
+ *
+ * @param descriptor a resource descriptor contains the initial parameters
+ * @param getResource a function to resolve resources (e.g., tools) by name and type
+ * @throws IllegalArgumentException if endpoint is null or empty
+ */
+ public AzureAIChatModelConnection(
+ ResourceDescriptor descriptor, BiFunction<String, ResourceType, Resource> getResource) {
+ super(descriptor, getResource);
+
+ String endpoint = descriptor.getArgument("endpoint");
+ String apiKey = descriptor.getArgument("apiKey");
+ if (endpoint == null || endpoint.isEmpty()) {
+ throw new IllegalArgumentException("endpoint should not be null or empty.");
+ }
+ this.client =
+ new ChatCompletionsClientBuilder()
+ .credential(new AzureKeyCredential(apiKey))
+ .endpoint(endpoint)
+ .buildClient();
+ }
+
+ @SuppressWarnings("unchecked")
+ private List<ChatCompletionsToolDefinition> convertToAzureAITools(List<Tool> tools) {
+ final ObjectMapper mapper = new ObjectMapper();
+ final List<ChatCompletionsToolDefinition> azureAITools = new ArrayList<>();
+ try {
+ for (Tool tool : tools) {
+ final Map<String, Object> schema =
+ mapper.readValue(
+ tool.getMetadata().getInputSchema(), new TypeReference<>() {});
+
+ final FunctionDefinition functionDef =
+ new FunctionDefinition(tool.getName())
+ .setDescription(tool.getDescription())
+ .setParameters(BinaryData.fromObject(schema));
+
+ azureAITools.add(new ChatCompletionsFunctionToolDefinition(functionDef));
+ }
+ return azureAITools;
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ private ChatRequestMessage convertToChatRequestMessage(ChatMessage message) {
+ final String content = message.getContent();
+ final MessageRole role = message.getRole();
+ switch (role) {
+ case SYSTEM:
+ return new ChatRequestSystemMessage(content);
+ case USER:
+ return new ChatRequestUserMessage(content);
+ case ASSISTANT:
+ return new ChatRequestAssistantMessage(content);
Review Comment:
Here, only the `content` of `ChatRequestAssistantMessage` is set, but no `toolCalls` are set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on PR #290:
URL: https://github.com/apache/flink-agents/pull/290#issuecomment-3484546150
Hi, @tsaiggo, this pr looks good to me.
Since merging a PR requires apache committer privileges, @xintongsong could you take a look at this pr at your convenience?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "tsaiggo (via GitHub)" <gi...@apache.org>.
tsaiggo commented on PR #290:
URL: https://github.com/apache/flink-agents/pull/290#issuecomment-3480836202
Looks like the issue of python-related test case has been self-recovered.
lease let me know if you have any further comments or feedback—even nitpicks are welcome. If not, is this PR good to go? @wenjin272
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "tsaiggo (via GitHub)" <gi...@apache.org>.
tsaiggo commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2490510667
##########
examples/pom.xml:
##########
@@ -64,6 +64,11 @@ under the License.
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-agents-integrations-chat-models-azureai</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
added the `azureaichatmodel` to the `examples` module because I intended it to be our first example of a cloud-hosted model. My original plan was to add specific use cases for cloud-hosted models, but I haven't come up with any good ideas for that yet, so it's temporarily on hold. If you think this addition should be removed for now, I'm happy to take care of that.
##########
examples/pom.xml:
##########
@@ -64,6 +64,11 @@ under the License.
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-agents-integrations-chat-models-azureai</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
added the `azureaichatmodel` to the `examples` module because I intended it to be our first example of a cloud-hosted model. My original plan was to add specific use cases for cloud-hosted models, but I haven't come up with any good ideas for that yet, so it's temporarily on hold. If you think this addition should be removed for now, I'm happy to take care of that. @xintongsong
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on code in PR #290:
URL: https://github.com/apache/flink-agents/pull/290#discussion_r2485412026
##########
integrations/chat-models/azureai/src/main/java/org/apache/flink/agents/integrations/chatmodels/azureai/AzureAIChatModelSetup.java:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.agents.integrations.chatmodels.azureai;
+
+import org.apache.flink.agents.api.chat.model.BaseChatModelSetup;
+import org.apache.flink.agents.api.resource.Resource;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.apache.flink.agents.api.resource.ResourceType;
+
+/**
+ * A chat model integration for Azure AI Chat Completions service.
+ *
+ * <p>This implementation adapts the generic Flink Agents chat model interface to the Azure AI Chat
+ * Completions API.
+ *
+ * <p>See also {@link BaseChatModelSetup} for the common resource abstractions and lifecycle.
+ *
+ * <p>Example usage:
+ *
+ * <pre>{@code
+ * public class MyAgent extends Agent {
+ * // Register the chat model setup via @ChatModelSetup metadata.
+ * @ChatModelSetup
+ * public static ResourceDesc azureAI() {
+ * return ResourceDescriptor.Builder.newBuilder(AzureAIChatModelSetup.class.getName())
+ * .addInitialArgument("model", "<your-azure-ai-model-name>")
+ * .addInitialArgument("prompt", "<your-prompt-template>")
+ * .addInitialArgument("tools", "<your-tool-list>")
+ * .addInitialArgument("key", "<your-azure-ai-key>")
+ * .addInitialArgument("endpoint", "<your-azure-ai-endpoint>")
+ * .build();
+ * }
+ * }
+ * }</pre>
+ */
+public class AzureAIChatModelSetup extends BaseChatModelSetup {
+
+ public AzureAIChatModelSetup(
+ ResourceDescriptor descriptor,
+ java.util.function.BiFunction<String, ResourceType, Resource> getResource) {
+ super(descriptor, getResource);
+ }
+
+ // For any other specific parameters, please refer to ChatCompletionsOptions @Override
Review Comment:
The `@Override` annotation is commented out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on PR #290:
URL: https://github.com/apache/flink-agents/pull/290#issuecomment-3471244251
Hi, @tsaiggo, thanks for your work. I will take a look today.
I verified the question of ollama test you mentioned, it is generated after we minor refactor the `OllamaChatModelConnection` in #279 but forgot to adapt the ollama integration test. I fix it in #291.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "tsaiggo (via GitHub)" <gi...@apache.org>.
tsaiggo commented on PR #290:
URL: https://github.com/apache/flink-agents/pull/290#issuecomment-3474327722
pushed new changes, please re-review this PR, thx @wenjin272
btw, there might be something wrong with the python-related test case, can you help retry the failed job? I have no access to do this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [feature][java][chatmodels] Add Azure AI chat model integration [flink-agents]
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong merged PR #290:
URL: https://github.com/apache/flink-agents/pull/290
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org