You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by "Thundercloud12 (via GitHub)" <gi...@apache.org> on 2026/09/26 13:59:31 UTC
[I] [Bug][Grafana/Dashboards] Multi-AI Tool Comparison dashboard omits Claude Code and has incomplete panel layout [devlake]
Thundercloud12 opened a new issue, #9170:
URL: https://github.com/apache/devlake/issues/9170
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues.
### What happened
The Multi-AI Tool Comparison dashboard (multi-ai-comparison.json)
benchmarks AI tool adoption and activity across organizations. In
PR #9017, the dashboard was expanded from Copilot and Kiro to
also include Cursor (Multi-AI Tool Comparison: Copilot vs Kiro vs
Cursor).
However, Claude Code (introduced in PR #8808 via
backend/plugins/claude_code) is completely missing from this
multi-tool comparison dashboard:
1. Missing Metrics & Tool Incomplete Comparison:
• Weekly Active Users: Compares Kiro, Copilot, and Cursor,
but omits Claude Code active users (which are collected in
_tool_claude_code_activity_summary and
_tool_claude_code_user_activity).
• Lines of Code (LOC) Added: Aggregates Kiro returned lines,
Copilot LOC added, and Cursor LOC added, but omits Claude
Code cc_lines_added.
• Acceptance Rate Comparison: Compares Copilot and Cursor
acceptance rates, but does not include Claude Code tool
action acceptance rate ((edit + write + multiedit accepted) /
total).
• Connection & Scope Variables: Templating list lacks
variables for Claude Code connections and scopes.
2. Unbalanced Grid Layout:
• In the Assistant Activity row, panel 13 (Cursor: Tab
Completions) is assigned grid position {h: 8, w: 12, x: 0, y:
21}.
• The adjacent right grid position {h: 8, w: 12, x: 12, y:
21} is left completely empty, resulting in a blank gap in the
dashboard before the next row begins at y: 29.
3. Dashboard Parity:
• In addition, the PostgreSQL version
(multi-ai-comparison.json) was never updated when Cursor was
added, so it remains out of sync and lacks both Cursor and
Claude Code.
### What do you expect to happen
1. multi-ai-comparison.json (both MySQL and PostgreSQL variants)
should include Claude Code metrics alongside Copilot, Kiro, and
Cursor:
• WAU: Include Claude Code active users from
_tool_claude_code_user_activity or
_tool_claude_code_activity_summary.
• Assistant Activity: Fill the empty slot at {x: 12, y: 21,
w: 12, h: 8} with a panel for Claude Code tool activity (e.g.
Claude Code: Tool Actions (Accepted vs Rejected) or Sessions
& Tool Modifications).
• Lines of Code Added: Include SUM(cc_lines_added) from
_tool_claude_code_user_activity.
• Acceptance Rate: Include Claude Code tool acceptance rate
(edit_tool_accepted, write_tool_accepted, etc. vs rejected).
• Templating Variables: Add claude_code_connection_id and
claude_code_scope_id template dropdowns.
2. The "Semantic boundary" documentation at the top of the
dashboard should be updated to clarify the boundary for Claude
Code (i.e. agentic tool actions and CLI sessions vs. inline IDE
autocompletions).
### How to reproduce
1. Deploy DevLake with both cursor and claude_code plugins
enabled and collect data for both.
2. Open Grafana and navigate to the Multi-AI Tool Comparison
dashboard (/d/multi_ai_comparison).
3. Notice that:
• There are no connection/scope filters or panels for Claude
Code.
• There is a 12-column empty gap to the right of the Cursor
Tab Completions panel in the Assistant Activity row.
### Anything else
_No response_
### Version
main (commit a18194b96b5b332a60a74b23d51bacbb4262234d)
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][Grafana/Dashboards] Multi-AI Tool Comparison dashboard omits Claude Code and has incomplete panel layout [devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh closed issue #9170: [Bug][Grafana/Dashboards] Multi-AI Tool Comparison dashboard omits Claude Code and has incomplete panel layout
URL: https://github.com/apache/devlake/issues/9170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][Grafana/Dashboards] Multi-AI Tool Comparison dashboard omits Claude Code and has incomplete panel layout [devlake]
Posted by "Thundercloud12 (via GitHub)" <gi...@apache.org>.
Thundercloud12 commented on issue #9170:
URL: https://github.com/apache/devlake/issues/9170#issuecomment-5846867308
@klesh was going through ai dashboard code to understand that plugin and devlake in general that is when i noticed this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org