You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "peachisai (via GitHub)" <gi...@apache.org> on 2026/07/10 04:47:35 UTC
[PR] add the genai evaluation record graphql [skywalking-query-protocol]
peachisai opened a new pull request, #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163
<img width="2479" height="1352" alt="image" src="https://github.com/user-attachments/assets/5dbd334e-1609-4cb5-bffb-370bc38d0700" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163#discussion_r3762949687
##########
gen-ai-evaluation-record.graphqls:
##########
@@ -0,0 +1,83 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+type GenAIEvaluationRecords {
+ genAIEvaluationRecordList: [GenAIEvaluationRecord!]!
+ debuggingTrace: DebuggingTrace
+}
+
+type GenAIEvaluationRecord {
+ traceId: String
+ serviceId: String
+ serviceName: String
+ providerId: String
+ providerName: String
+ modelId: String
+ modelName: String
+ operationName: String
+ scoreValue: Float
+ segmentId: String
+ spanId: String
+ spanType: String
+ taskName: String
+ valueType: GenAIEvaluationValueType
Review Comment:
We need comments/documents about which fields are value holders for each type.
And we should put them together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163#discussion_r3764128296
##########
gen-ai-evaluation-record.graphqls:
##########
@@ -0,0 +1,89 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+type GenAIEvaluationRecords {
+ genAIEvaluationRecordList: [GenAIEvaluationRecord!]!
+ debuggingTrace: DebuggingTrace
+}
+
+type GenAIEvaluationRecord {
+ traceId: String
+ serviceId: String
+ serviceName: String
+ providerId: String
+ providerName: String
+ modelId: String
+ modelName: String
+ operationName: String
+ # The value type determines which field contains the evaluation value.
+ # SCORE and BOOLEAN use scoreValue; STRING and JSON use value.
+ valueType: GenAIEvaluationValueType
+ # Numeric value stored in evaNumberValue. It is populated for SCORE and BOOLEAN.
+ scoreValue: Float
Review Comment:
```suggestion
scoreValue: Long
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163#discussion_r3665794047
##########
gen-ai-evaluation-record.graphqls:
##########
@@ -0,0 +1,59 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+type GenAIEvaluationRecords {
+ genAIEvaluationRecordList: [GenAIEvaluationRecord!]!
+ errorReason: String
+ debuggingTrace: DebuggingTrace
+}
+
+type GenAIEvaluationRecord {
+ traceId: String
+ serviceId: String
+ serviceInstanceId: String
+ segmentId: String
+ spanId: String
+ spanType: String
+ taskName: String
+ valueType: String
+ value: String
+ evaluationLevel: String
+ reason: String
+ judgeModel: String
+ evaluationTime: Long
+}
+
+input GenAIEvaluationRecordTag {
+ key: String!
+ value: String
+}
+
+input GenAIEvaluationRecordQueryCondition {
+ serviceId: ID
+ serviceInstanceId: ID
+ relatedTrace: TraceScopeCondition
+ queryDuration: Duration
+ paging: Pagination!
+ tags: [GenAIEvaluationRecordTag!]
+ queryOrder: Order
+}
+
+extend type Query {
+ supportGenAIEvaluationRecordQueryByKeywords: Boolean!
Review Comment:
Is this for whole-text-searching powered by Elasticsearch only?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng merged PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163#discussion_r3762946169
##########
gen-ai-evaluation-record.graphqls:
##########
@@ -0,0 +1,83 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+type GenAIEvaluationRecords {
+ genAIEvaluationRecordList: [GenAIEvaluationRecord!]!
+ debuggingTrace: DebuggingTrace
+}
+
+type GenAIEvaluationRecord {
+ traceId: String
+ serviceId: String
+ serviceName: String
+ providerId: String
+ providerName: String
+ modelId: String
+ modelName: String
+ operationName: String
+ scoreValue: Float
+ segmentId: String
+ spanId: String
+ spanType: String
+ taskName: String
+ valueType: GenAIEvaluationValueType
+ value: String
+ evaluationLevel: String
+ reason: String
+ judgeModel: String
+ evaluationTime: Long
+}
+
+enum GenAIEvaluationRecordSortBy {
+ EVALUATION_TIME
+ SCORE_VALUE
+}
+
+enum GenAIEvaluationValueType {
+ SCORE
+ BOOLEAN
+ STRING
+ JSON
+}
+
+input GenAIEvaluationRecordTag {
+ key: String!
+ value: String
+}
+
+input GenAIEvaluationRecordQueryCondition {
+ serviceId: ID
+ providerId: ID
+ modelId: ID
+ valueType: GenAIEvaluationValueType
+ minScore: Float
+ maxScore: Float
Review Comment:
Please use Long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "peachisai (via GitHub)" <gi...@apache.org>.
peachisai commented on code in PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163#discussion_r3670574976
##########
gen-ai-evaluation-record.graphqls:
##########
@@ -0,0 +1,59 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+type GenAIEvaluationRecords {
+ genAIEvaluationRecordList: [GenAIEvaluationRecord!]!
+ errorReason: String
+ debuggingTrace: DebuggingTrace
+}
+
+type GenAIEvaluationRecord {
+ traceId: String
+ serviceId: String
+ serviceInstanceId: String
+ segmentId: String
+ spanId: String
+ spanType: String
+ taskName: String
+ valueType: String
+ value: String
+ evaluationLevel: String
+ reason: String
+ judgeModel: String
+ evaluationTime: Long
+}
+
+input GenAIEvaluationRecordTag {
+ key: String!
+ value: String
+}
+
+input GenAIEvaluationRecordQueryCondition {
+ serviceId: ID
+ serviceInstanceId: ID
+ relatedTrace: TraceScopeCondition
+ queryDuration: Duration
+ paging: Pagination!
+ tags: [GenAIEvaluationRecordTag!]
+ queryOrder: Order
+}
+
+extend type Query {
+ supportGenAIEvaluationRecordQueryByKeywords: Boolean!
Review Comment:
> Is this for whole-text-searching powered by Elasticsearch only?
Yes. This is intended for full-text search and will be supported only by Elasticsearch. It hasn't been implemented yet due to time constraints, so this field currently always returns false. I kept it for planned future support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163#discussion_r3764126405
##########
gen-ai-evaluation-record.graphqls:
##########
@@ -0,0 +1,89 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+type GenAIEvaluationRecords {
+ genAIEvaluationRecordList: [GenAIEvaluationRecord!]!
+ debuggingTrace: DebuggingTrace
+}
+
+type GenAIEvaluationRecord {
+ traceId: String
+ serviceId: String
+ serviceName: String
+ providerId: String
+ providerName: String
+ modelId: String
+ modelName: String
+ operationName: String
+ # The value type determines which field contains the evaluation value.
+ # SCORE and BOOLEAN use scoreValue; STRING and JSON use value.
+ valueType: GenAIEvaluationValueType
+ # Numeric value stored in evaNumberValue. It is populated for SCORE and BOOLEAN.
+ scoreValue: Float
+ # String or JSON value stored in evalStringValue. It is populated for STRING and JSON.
+ value: String
+ segmentId: String
+ spanId: String
+ spanType: String
+ taskName: String
+ evaluationLevel: String
+ reason: String
+ judgeModel: String
+ evaluationTime: Long
+}
+
+enum GenAIEvaluationRecordSortBy {
+ EVALUATION_TIME
+ SCORE_VALUE
+}
+
+enum GenAIEvaluationValueType {
+ SCORE
+ BOOLEAN
+ STRING
+ JSON
+}
+
+input GenAIEvaluationRecordTag {
+ key: String!
+ value: String
+}
+
+input GenAIEvaluationRecordQueryCondition {
+ serviceId: ID
+ providerId: ID
+ modelId: ID
+ # Filter value_type first when selecting a specific evaluation kind.
+ valueType: GenAIEvaluationValueType
+ # Bounds for evaNumberValue, used with SCORE or BOOLEAN value types.
+ minScore: Long
+ maxScore: Long
Review Comment:
```suggestion
# Bounds for evaNumberValue, used with SCORE or BOOLEAN value types.
minScore: Long
maxScore: Long
```
We should say, we only support query by Score range. And we should determine what is our range?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "peachisai (via GitHub)" <gi...@apache.org>.
peachisai commented on PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163#issuecomment-5282334499
@wu-sheng
<img width="1956" height="1218" alt="image" src="https://github.com/user-attachments/assets/54f8aa0a-6ed8-4aba-897d-09705d40572b" />
<img width="1941" height="987" alt="image" src="https://github.com/user-attachments/assets/bd80101b-bfbb-45d8-8d7a-d39aba8d89a4" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163#discussion_r3665894418
##########
gen-ai-evaluation-record.graphqls:
##########
@@ -0,0 +1,59 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+type GenAIEvaluationRecords {
+ genAIEvaluationRecordList: [GenAIEvaluationRecord!]!
+ errorReason: String
+ debuggingTrace: DebuggingTrace
+}
+
+type GenAIEvaluationRecord {
+ traceId: String
+ serviceId: String
+ serviceInstanceId: String
Review Comment:
We need names for them to visualize. How did you show that in the page? decoding on UI side seems to be not correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] add the genai evaluation record graphql [skywalking-query-protocol]
Posted by "peachisai (via GitHub)" <gi...@apache.org>.
peachisai commented on code in PR #163:
URL: https://github.com/apache/skywalking-query-protocol/pull/163#discussion_r3670574976
##########
gen-ai-evaluation-record.graphqls:
##########
@@ -0,0 +1,59 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+type GenAIEvaluationRecords {
+ genAIEvaluationRecordList: [GenAIEvaluationRecord!]!
+ errorReason: String
+ debuggingTrace: DebuggingTrace
+}
+
+type GenAIEvaluationRecord {
+ traceId: String
+ serviceId: String
+ serviceInstanceId: String
+ segmentId: String
+ spanId: String
+ spanType: String
+ taskName: String
+ valueType: String
+ value: String
+ evaluationLevel: String
+ reason: String
+ judgeModel: String
+ evaluationTime: Long
+}
+
+input GenAIEvaluationRecordTag {
+ key: String!
+ value: String
+}
+
+input GenAIEvaluationRecordQueryCondition {
+ serviceId: ID
+ serviceInstanceId: ID
+ relatedTrace: TraceScopeCondition
+ queryDuration: Duration
+ paging: Pagination!
+ tags: [GenAIEvaluationRecordTag!]
+ queryOrder: Order
+}
+
+extend type Query {
+ supportGenAIEvaluationRecordQueryByKeywords: Boolean!
Review Comment:
> Is this for whole-text-searching powered by Elasticsearch only?
Yes. This is intended for Elasticsearch-powered full-text search. It hasn't been implemented yet, so the capability field currently always returns false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org