You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by "stringl1l1l1l (via GitHub)" <gi...@apache.org> on 2026/03/05 13:37:13 UTC
[PR] refactor(ai): componentize runtime and configuration with schema-validated loading [dubbo-admin]
stringl1l1l1l opened a new pull request, #1421:
URL: https://github.com/apache/dubbo-admin/pull/1421
This PR restructures the AI module around a component-based runtime, adds schema-driven configuration loading/validation, and aligns test coverage with the new architecture.
## What Changed
1. Architecture refactor
- Migrated AI domains into `ai/component/*` and unified startup through a factory-based runtime.
- Standardized component lifecycle orchestration (`Validate -> Init -> Start`).
2. Configuration system upgrade
- Added a schema-aware loader with strict YAML decoding and JSON Schema validation.
- Added schema assets under `ai/schema/json` and unified component config entrypoints via `ai/config.yaml`.
3. CLI and workflow alignment
- Added/updated indexing CLI flow to use the new config/component model.
- Updated environment/config conventions (including `SCHEMA_DIR` and provider key usage).
4. Test consolidation
- Reorganized and expanded tests for config loader, runtime orchestration, component validation, and key workflows.
## Breaking Changes
- Startup is now config-driven (`-config`, default `config.yaml`) rather than legacy server flags.
- Config files are strictly validated; unknown or structurally invalid fields now fail fast.
- Component YAML locations and wiring are now expected via `ai/config.yaml`.
## Validation
```bash
go test -race -v ./config/... ./runtime/... ./component/... ./test/... ./cmd/...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [PR] refactor(ai): componentize runtime and configuration with schema-validated loading [dubbo-admin]
Posted by "robocanic (via GitHub)" <gi...@apache.org>.
robocanic commented on PR #1421:
URL: https://github.com/apache/dubbo-admin/pull/1421#issuecomment-4062515982
Merge first and improve later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [PR] refactor(ai): componentize runtime and configuration with schema-validated loading [dubbo-admin]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #1421:
URL: https://github.com/apache/dubbo-admin/pull/1421#discussion_r2901808009
##########
ai/cmd/index.go:
##########
@@ -0,0 +1,177 @@
+package main
+
+import (
+ "context"
+ compRag "dubbo-admin-ai/component/rag"
+ appconfig "dubbo-admin-ai/config"
+ "flag"
+ "fmt"
+ "log"
+ "os"
+ "path/filepath"
+ "strings"
+
+ "github.com/firebase/genkit/go/core/api"
+ "github.com/firebase/genkit/go/genkit"
+ "github.com/firebase/genkit/go/plugins/compat_oai"
+ "github.com/firebase/genkit/go/plugins/googlegenai"
+ "github.com/firebase/genkit/go/plugins/pinecone"
+ "github.com/openai/openai-go/option"
+)
+
+type IndexCommand struct {
+ Directory string
+ ConfigPath string
+}
+
+func main() {
+ cmd := parseFlags()
+
+ if err := validateCommand(cmd); err != nil {
+ log.Fatalf("Validation error: %v", err)
+ }
+
+ if err := executeIndexing(cmd); err != nil {
+ log.Fatalf("Indexing failed: %v", err)
+ }
+
+ fmt.Println("Indexing completed successfully")
+}
+
+func parseFlags() *IndexCommand {
+ cmd := &IndexCommand{}
+
+ flag.StringVar(&cmd.Directory, "dir", "/Users/liwener/programming/ospp/dubbo-admin/ai/reference/k8s_docs/concepts", "Directory to index (required)")
Review Comment:
The `-dir` flag default is a developer-specific absolute path ("/Users/liwener/..."), which will break for other environments. Prefer an empty default (and require the flag) or a repo-relative path (e.g., "./reference/..."), and document it in the usage string.
```suggestion
flag.StringVar(&cmd.Directory, "dir", "", "Directory to index (required, no default)")
```
##########
ai/component/rag/indexer.go:
##########
@@ -0,0 +1,203 @@
+package rag
+
+import (
+ "context"
+
+ "dubbo-admin-ai/utils"
+ "fmt"
+ "sync"
+
+ "github.com/cloudwego/eino/components/indexer"
+ "github.com/cloudwego/eino/schema"
+ "github.com/firebase/genkit/go/ai"
+ "github.com/firebase/genkit/go/core"
+ "github.com/firebase/genkit/go/genkit"
+ "github.com/firebase/genkit/go/plugins/localvec"
+ "github.com/firebase/genkit/go/plugins/pinecone"
+)
+
+// --- Indexer ---
+type PineconeIndexer struct {
+ g *genkit.Genkit
+ embedder string
+ target string
+ batchSz int
+ mu sync.Mutex
+ docstore map[string]*pinecone.Docstore // keyed by target index
+}
+
+func newPineconeIndexer(g *genkit.Genkit, embedderModel string, targetIndex string, batchSize int) *PineconeIndexer {
+ return &PineconeIndexer{
+ g: g,
+ embedder: embedderModel,
+ target: targetIndex,
+ batchSz: batchSize,
+ }
+}
+
+func (idx *PineconeIndexer) Store(ctx context.Context, docs []*schema.Document, opts ...indexer.Option) ([]string, error) {
+ // Handle options
+ implOpts := indexer.GetImplSpecificOptions(&CommonIndexerOptions{}, opts...)
+ namespace := implOpts.Namespace
+ effectiveTarget := idx.target
+ if implOpts.TargetIndex != nil && *implOpts.TargetIndex != "" {
+ effectiveTarget = *implOpts.TargetIndex
+ }
+
+ // TODO(indexer, 2026-02-24): Validate namespace if needed for multi-tenancy support
+ // Initialize indexer docstore for this target if not already done
+ idx.mu.Lock()
+ if idx.docstore == nil {
+ idx.docstore = make(map[string]*pinecone.Docstore)
+ }
+ docstore := idx.docstore[effectiveTarget]
+ idx.mu.Unlock()
+ if docstore == nil {
+ embedder := genkit.LookupEmbedder(idx.g, idx.embedder)
+ if embedder == nil {
+ return nil, fmt.Errorf("failed to find embedder %s", idx.embedder)
+ }
+
+ // Configure Pinecone connection
+ pineconeConfig := pinecone.Config{
+ IndexID: effectiveTarget,
+ Embedder: embedder,
+ }
+
+ newDocstore, _, err := pinecone.DefineRetriever(ctx, idx.g,
+ pineconeConfig,
+ &ai.RetrieverOptions{
+ Label: effectiveTarget,
+ ConfigSchema: core.InferSchemaMap(pinecone.PineconeRetrieverOptions{}),
+ })
+ if err != nil {
+ return nil, fmt.Errorf("failed to setup retriever for indexer: %w", err)
+ }
+
+ idx.mu.Lock()
+ if idx.docstore == nil {
+ idx.docstore = make(map[string]*pinecone.Docstore)
+ }
+ if idx.docstore[effectiveTarget] == nil {
+ idx.docstore[effectiveTarget] = newDocstore
+ }
+ docstore = idx.docstore[effectiveTarget]
+ idx.mu.Unlock()
+ }
+
+ // Convert to Genkit documents
+ genkitDocs := utils.ToGenkitDocuments(docs)
+
+ // Index in batches
+ batchSize := idx.batchSz
+ if implOpts.BatchSize != nil && *implOpts.BatchSize > 0 {
+ batchSize = *implOpts.BatchSize
+ }
+ if batchSize <= 0 {
+ return nil, fmt.Errorf("batch size must be positive")
+ }
+ for i := 0; i < len(genkitDocs); i += batchSize {
+ end := min(i+batchSize, len(genkitDocs))
+ batch := genkitDocs[i:end]
+ if err := pinecone.Index(ctx, batch, docstore, namespace); err != nil {
+ return nil, fmt.Errorf("failed to index documents batch %d-%d: %w", i+1, end, err)
+ }
+ }
+
+ return nil, nil
+}
Review Comment:
PineconeIndexer.Store returns `(nil, nil)` after successful indexing. The indexer contract expects IDs for the stored documents (DevIndexer returns them), and returning nil can break callers that rely on IDs or interpret nil as “nothing stored”. Consider returning the input document IDs (or IDs returned by the backend if available).
##########
ai/runtime/runtime.go:
##########
@@ -0,0 +1,237 @@
+package runtime
+
+import (
+ "context"
+ "dubbo-admin-ai/config"
+ "fmt"
+ "log/slog"
+ "sync"
+
+ "github.com/firebase/genkit/go/genkit"
+ "gopkg.in/yaml.v3"
+)
+
+// Component defines the interface for all components
+type Component interface {
+ Name() string
+ Validate() error
+ Init(*Runtime) error
+ Start() error
+ Stop() error
+}
+
+// ComponentFactory is the function type for creating components
+type ComponentFactory func(config *yaml.Node) (Component, error)
+
+var (
+ gloRuntime *Runtime = nil
+)
+
+func NewRuntime() *Runtime {
+ return &Runtime{
+ factories: make(map[string]ComponentFactory),
+ factoryOrder: make([]string, 0),
+ genkitOptions: make([]genkit.GenkitOption, 0),
+ }
+}
+
+func Bootstrap(configFile string, registerFn func(rt *Runtime)) (*Runtime, error) {
+ gloRuntime = NewRuntime()
+
+ // Register component factories
+ if registerFn != nil {
+ registerFn(gloRuntime)
+ }
+
+ // Create config loader and load all configurations
+ loader := config.NewLoader(configFile)
+ loadedCfg, err := loader.Load()
+ if err != nil {
+ return nil, fmt.Errorf("failed to load config: %w", err)
+ }
+
+ // Create component instances
+ instances, err := gloRuntime.createComponents(loadedCfg)
+ if err != nil {
+ return nil, fmt.Errorf("failed to create components: %w", err)
+ }
+
+ // Initialize components in dependency order, which is the order of factory registration.
+ for _, comp := range instances {
+ if err := comp.Validate(); err != nil {
+ return nil, fmt.Errorf("failed to validate %s: %w", comp.Name(), err)
+ }
+
+ if err := comp.Init(gloRuntime); err != nil {
+ return nil, fmt.Errorf("failed to init %s: %w", comp.Name(), err)
+ }
+ gloRuntime.Components.Store(comp.Name(), comp)
+ }
Review Comment:
Components are stored in the runtime by `comp.Name()`. Since many components return a fixed name (e.g., "agent"), this will overwrite entries when `components.<name>` in config.yaml is an array (Loader generates names like `agent-0`, `agent-1`). To support multiple instances, store components under the config key (name from LoadedConfig.Components) and/or allow components to be instantiated with a unique name.
##########
ai/test/models.md:
##########
@@ -0,0 +1,146 @@
+# 模型可用性测试指南
+
+本指南说明如何测试 `config/models.yaml` 中配置的所有模型是否可用。
+
+## 测试方法
+
+### 方法1：使用现有测试（推荐）
+
+AI模块已经有一个简单的文本生成测试，可以快速验证默认模型：
+
+```bash
+cd /Users/liwener/programming/ospp/dubbo-admin/ai
+
+# 设置API密钥
+export DASHSCOPE_API_KEY="your_qwen_api_key"
+
+# 运行测试
+go test -v ./test/ -run TestTextGeneration
+```
Review Comment:
The guide uses absolute paths (e.g., `cd /Users/.../ai`) which are not portable. Prefer repo-relative commands (e.g., `cd ai`) so the documentation works for all contributors.
##########
ai/runtime/runtime.go:
##########
@@ -0,0 +1,237 @@
+package runtime
+
+import (
+ "context"
+ "dubbo-admin-ai/config"
+ "fmt"
+ "log/slog"
+ "sync"
+
+ "github.com/firebase/genkit/go/genkit"
+ "gopkg.in/yaml.v3"
+)
+
+// Component defines the interface for all components
+type Component interface {
+ Name() string
+ Validate() error
+ Init(*Runtime) error
+ Start() error
+ Stop() error
+}
+
+// ComponentFactory is the function type for creating components
+type ComponentFactory func(config *yaml.Node) (Component, error)
+
+var (
+ gloRuntime *Runtime = nil
+)
+
+func NewRuntime() *Runtime {
+ return &Runtime{
+ factories: make(map[string]ComponentFactory),
+ factoryOrder: make([]string, 0),
+ genkitOptions: make([]genkit.GenkitOption, 0),
+ }
+}
+
+func Bootstrap(configFile string, registerFn func(rt *Runtime)) (*Runtime, error) {
+ gloRuntime = NewRuntime()
+
+ // Register component factories
+ if registerFn != nil {
+ registerFn(gloRuntime)
+ }
+
+ // Create config loader and load all configurations
+ loader := config.NewLoader(configFile)
+ loadedCfg, err := loader.Load()
+ if err != nil {
+ return nil, fmt.Errorf("failed to load config: %w", err)
+ }
+
+ // Create component instances
+ instances, err := gloRuntime.createComponents(loadedCfg)
+ if err != nil {
+ return nil, fmt.Errorf("failed to create components: %w", err)
+ }
+
+ // Initialize components in dependency order, which is the order of factory registration.
+ for _, comp := range instances {
+ if err := comp.Validate(); err != nil {
+ return nil, fmt.Errorf("failed to validate %s: %w", comp.Name(), err)
+ }
+
+ if err := comp.Init(gloRuntime); err != nil {
+ return nil, fmt.Errorf("failed to init %s: %w", comp.Name(), err)
+ }
+ gloRuntime.Components.Store(comp.Name(), comp)
+ }
+
+ // Start all loaded components
+ gloRuntime.Components.Range(func(key, value any) bool {
+ comp := value.(Component)
+ if err := comp.Start(); err != nil {
+ return false
+ }
+ return true
+ })
Review Comment:
Bootstrap starts components via sync.Map.Range, which has non-deterministic iteration order and discards the Start() error (the callback returns false but the error is not surfaced). This can leave the runtime partially started while main logs success. Consider starting components in the same deterministic order as initialization (e.g., iterate the `instances` slice) and return the first start error from Bootstrap (and/or track started components for rollback).
##########
ai/cmd/index.go:
##########
@@ -0,0 +1,177 @@
+package main
+
+import (
+ "context"
+ compRag "dubbo-admin-ai/component/rag"
+ appconfig "dubbo-admin-ai/config"
+ "flag"
+ "fmt"
+ "log"
+ "os"
+ "path/filepath"
+ "strings"
+
+ "github.com/firebase/genkit/go/core/api"
+ "github.com/firebase/genkit/go/genkit"
+ "github.com/firebase/genkit/go/plugins/compat_oai"
+ "github.com/firebase/genkit/go/plugins/googlegenai"
+ "github.com/firebase/genkit/go/plugins/pinecone"
+ "github.com/openai/openai-go/option"
+)
+
+type IndexCommand struct {
+ Directory string
+ ConfigPath string
+}
+
+func main() {
+ cmd := parseFlags()
+
+ if err := validateCommand(cmd); err != nil {
+ log.Fatalf("Validation error: %v", err)
+ }
+
+ if err := executeIndexing(cmd); err != nil {
+ log.Fatalf("Indexing failed: %v", err)
+ }
+
+ fmt.Println("Indexing completed successfully")
+}
+
+func parseFlags() *IndexCommand {
+ cmd := &IndexCommand{}
+
+ flag.StringVar(&cmd.Directory, "dir", "/Users/liwener/programming/ospp/dubbo-admin/ai/reference/k8s_docs/concepts", "Directory to index (required)")
+ flag.StringVar(&cmd.ConfigPath, "config", "component/rag/rag.yaml", "Configuration file path")
+
+ flag.Parse()
+ return cmd
+}
+
+func validateCommand(cmd *IndexCommand) error {
+ if cmd.Directory == "" {
+ return fmt.Errorf("directory parameter is required")
+ }
+
+ // Check if directory exists
+ info, err := os.Stat(cmd.Directory)
+ if err != nil {
+ return fmt.Errorf("failed to access directory %s: %w", cmd.Directory, err)
+ }
+
+ if !info.IsDir() {
+ return fmt.Errorf("%s is not a directory", cmd.Directory)
+ }
+
+ // Check if config file exists
+ if _, err := os.Stat(cmd.ConfigPath); err != nil {
+ return fmt.Errorf("failed to access config file %s: %w", cmd.ConfigPath, err)
+ }
+
+ return nil
+}
+
+func executeIndexing(cmd *IndexCommand) error {
+ ctx := context.Background()
+
+ // Load configuration from YAML file
+ cfg, err := loadRAGConfig(cmd.ConfigPath)
+ if err != nil {
+ return fmt.Errorf("failed to load config: %w", err)
+ }
+ if err := cfg.Validate(); err != nil {
+ return fmt.Errorf("invalid rag config: %w", err)
+ }
+
+ // Initialize Genkit registry independently
+ plugins := []api.Plugin{}
+ // Initialize plugins from environment variables
+ if key := os.Getenv("DASHSCOPE_API_KEY"); key != "" {
+ plugins = append(plugins, &compat_oai.OpenAICompatible{
+ Provider: "dashscope",
+ Opts: []option.RequestOption{
+ option.WithAPIKey(key),
+ option.WithBaseURL("https://dashscope.aliyuncs.com/compatible-mode/v1"),
+ },
+ })
+ }
+ if key := os.Getenv("GEMINI_API_KEY"); key != "" {
+ plugins = append(plugins, &googlegenai.GoogleAI{APIKey: key})
+ }
+ if key := os.Getenv("SILICONFLOW_API_KEY"); key != "" {
+ plugins = append(plugins, &compat_oai.OpenAICompatible{
+ Provider: "siliconflow",
+ Opts: []option.RequestOption{
+ option.WithAPIKey(key),
+ option.WithBaseURL("https://api.siliconflow.cn/v1"),
+ },
+ })
+ }
+ if key := os.Getenv("PINECONE_API_KEY"); key != "" {
+ plugins = append(plugins, &pinecone.Pinecone{APIKey: key})
+ }
+
+ g := genkit.Init(ctx, genkit.WithPlugins(plugins...))
+
+ // Build-time target index selection: default to directory name for this CLI
+ targetIndex := getNamespace("", cmd.Directory)
+
+ sys, err := compRag.BuildRAGFromSpec(ctx, g, cfg)
+ if err != nil {
+ return fmt.Errorf("failed to build RAG system: %w", err)
+ }
+
+ // Load documents via the built loader
+ docs, err := compRag.LoadDirectory(ctx, sys.Loader, cmd.Directory)
+ if err != nil {
+ return fmt.Errorf("failed to load directory: %w", err)
+ }
+
+ fmt.Printf("Loaded %d documents from %s\n", len(docs), cmd.Directory)
+
+ if len(docs) == 0 {
+ fmt.Printf("No supported documents found in directory: %s\n", cmd.Directory)
+ return nil
+ }
+
+ // Split documents (semantic/header or recursive based on config)
+ splitDocs, err := sys.Split(ctx, docs)
+ if err != nil {
+ return fmt.Errorf("failed to split documents: %w", err)
+ }
+
+ // Index documents (namespace is a per-call runtime parameter)
+ namespace := targetIndex
+ if _, err := sys.Index(ctx, namespace, splitDocs, compRag.WithIndexerTargetIndex(targetIndex)); err != nil {
+ return fmt.Errorf("failed to index documents: %w", err)
+ }
+
+ return nil
+}
+
+// getNamespace generates a namespace from the provided parameter or directory name
+func getNamespace(namespace, directory string) string {
+ if namespace != "" {
+ return namespace
+ }
+ return filepath.Base(strings.TrimSpace(directory))
+}
+
+// loadRAGConfig loads RAG configuration from a YAML file
+func loadRAGConfig(configPath string) (*compRag.RAGSpec, error) {
+ loader := appconfig.NewLoader("config.yaml")
+ componentCfg, err := loader.LoadComponent(configPath)
+ if err != nil {
Review Comment:
loadRAGConfig initializes the Loader with `config.NewLoader("config.yaml")`, so schema/config resolution becomes dependent on the current working directory rather than the provided `configPath`. This makes the CLI fragile when run outside the ai/ directory. Consider constructing the Loader with a config file path rooted at `filepath.Dir(configPath)` (or using absolute paths) so relative component paths and default schema path resolve reliably.
```suggestion
// Resolve the provided config path to an absolute path so that loading
// does not depend on the current working directory.
absConfigPath, err := filepath.Abs(configPath)
if err != nil {
return nil, fmt.Errorf("failed to resolve absolute path for config %s: %w", configPath, err)
}
// Construct the loader's config file path relative to the config's directory.
baseDir := filepath.Dir(absConfigPath)
loaderConfigPath := filepath.Join(baseDir, "config.yaml")
loader := appconfig.NewLoader(loaderConfigPath)
componentCfg, err := loader.LoadComponent(absConfigPath)
if err != nil {
```
##########
ai/schema/json/README.md:
##########
@@ -0,0 +1,19 @@
+# JSON Schema Index
+
+This directory contains JSON Schema definitions for YAML configs.
+
+- `main.schema.json`: root [`config.yaml`](/Users/liwener/.codex/worktrees/acbb/dubbo-admin/ai/config.yaml)
+- `logger.schema.json`: [`component/logger/logger.yaml`](/Users/liwener/.codex/worktrees/acbb/dubbo-admin/ai/component/logger/logger.yaml)
+- `memory.schema.json`: [`component/memory/memory.yaml`](/Users/liwener/.codex/worktrees/acbb/dubbo-admin/ai/component/memory/memory.yaml)
+- `models.schema.json`: [`component/models/models.yaml`](/Users/liwener/.codex/worktrees/acbb/dubbo-admin/ai/component/models/models.yaml)
+- `tools.schema.json`: [`component/tools/tools.yaml`](/Users/liwener/.codex/worktrees/acbb/dubbo-admin/ai/component/tools/tools.yaml)
+- `server.schema.json`: [`component/server/server.yaml`](/Users/liwener/.codex/worktrees/acbb/dubbo-admin/ai/component/server/server.yaml)
+- `rag.schema.json`: [`component/rag/rag.yaml`](/Users/liwener/.codex/worktrees/acbb/dubbo-admin/ai/component/rag/rag.yaml)
+- `agent.schema.json`: [`component/agent/agent.yaml`](/Users/liwener/.codex/worktrees/acbb/dubbo-admin/ai/component/agent/agent.yaml)
Review Comment:
This README links to schema/config files using absolute local filesystem paths under `/Users/...`, which won’t work for other contributors or in GitHub UI. Use repository-relative links (e.g., `../../config.yaml` or `/ai/config.yaml`) instead.
##########
ai/config/jsonschema.go:
##########
@@ -0,0 +1,175 @@
+package config
+
+import (
+ "encoding/json"
+ "fmt"
+ "os"
+ "path/filepath"
+ "strings"
+ "sync"
+
+ "github.com/xeipuuv/gojsonschema"
+)
+
+// schemaEngine manages the loading, caching, and compilation of JSON schemas.
+type schemaEngine struct {
+ baseDir string
+ mu sync.RWMutex
+ cache map[string]*gojsonschema.Schema
+ rootObjs map[string]map[string]any
+}
+
+// NewSchemaEngine creates a new schemaEngine with the specified base directory.
+func NewSchemaEngine(baseDir string) *schemaEngine {
+ return &schemaEngine{
+ baseDir: baseDir,
+ cache: make(map[string]*gojsonschema.Schema),
+ rootObjs: make(map[string]map[string]any),
+ }
+}
+
+// ApplyDefaultsAndValidate applies default values and validates.
+// This method modifies doc in-place by applying defaults, then validates it.
+//
+// Parameters:
+// - doc: The configuration document (WILL BE MODIFIED)
+// - schemaFile: The schema filename
+//
+// Returns:
+// - The modified doc (same map as input)
+// - Error if validation fails
+func (e *schemaEngine) ApplyDefaultsAndValidate(doc map[string]any, schemaFile string) (map[string]any, error) {
+ compiled, rootObj, err := e.loadSchema(schemaFile)
+ if err != nil {
+ return nil, err
+ }
+
+ // Apply defaults in-place (modifies doc)
+ applyDefaults(rootObj, rootObj, doc)
+
+ // Validate the result
+ if err := validateJSONSchema(compiled, doc); err != nil {
+ return nil, err
+ }
+
+ return doc, nil
+}
+
+func (e *schemaEngine) loadSchema(fileName string) (*gojsonschema.Schema, map[string]any, error) {
+ e.mu.RLock()
+ compiled, hasCached := e.cache[fileName]
+ rootObj, hasRoot := e.rootObjs[fileName]
+ e.mu.RUnlock()
+
+ if hasCached && hasRoot {
+ return compiled, rootObj, nil
+ }
+
+ fullPath := filepath.Join(e.baseDir, fileName)
+ raw, err := os.ReadFile(fullPath)
+ if err != nil {
+ return nil, nil, fmt.Errorf("structural error: failed to read schema file %s: %w", fileName, err)
+ }
+
+ var schemaObj map[string]any
+ if err := json.Unmarshal(raw, &schemaObj); err != nil {
+ return nil, nil, fmt.Errorf("structural error: failed to parse schema file %s: %w", fileName, err)
+ }
+
+ compiled, err = gojsonschema.NewSchema(gojsonschema.NewBytesLoader(raw))
+ if err != nil {
+ return nil, nil, fmt.Errorf("structural error: failed to compile schema file %s: %w", fileName, err)
+ }
+
+ e.mu.Lock()
+ e.cache[fileName] = compiled
+ e.rootObjs[fileName] = schemaObj
+ e.mu.Unlock()
+ return compiled, schemaObj, nil
+}
+
+func validateJSONSchema(compiled *gojsonschema.Schema, doc any) error {
+ docRaw, err := json.Marshal(doc)
+ if err != nil {
+ return fmt.Errorf("failed to marshal config for schema validation: %w", err)
+ }
+
+ result, err := compiled.Validate(gojsonschema.NewBytesLoader(docRaw))
+ if err != nil {
+ return fmt.Errorf("failed to validate schema: %w", err)
+ }
+ if result.Valid() {
+ return nil
+ }
+
+ errMsgs := make([]string, 0, len(result.Errors()))
+ for _, e := range result.Errors() {
+ field := e.Field()
+ if field == "(root)" || field == "" {
+ field = "root"
+ }
+ errMsgs = append(errMsgs, fmt.Sprintf("%s: %s", field, e.Description()))
+ }
+ return fmt.Errorf("structural error: %s", strings.Join(errMsgs, "; "))
+}
+
+// applyDefaults recursively applies default values from schema to value
+// Modifies value in-place
+func applyDefaults(root map[string]any, schema map[string]any, value any) {
+ resolved := resolveSchemaRef(root, schema)
+
+ switch v := value.(type) {
+ case map[string]any:
+ props, _ := resolved["properties"].(map[string]any)
+ for key, propVal := range props {
+ propSchema, ok := propVal.(map[string]any)
+ if !ok {
+ continue
+ }
+ propSchema = resolveSchemaRef(root, propSchema)
+
+ // Apply default value if property is missing
+ if _, exists := v[key]; !exists {
+ if defVal, hasDefault := propSchema["default"]; hasDefault {
+ v[key] = defVal
+ }
+ }
+
+ // Recursively apply defaults to nested properties
+ if child, exists := v[key]; exists {
+ applyDefaults(root, propSchema, child)
+ }
+ }
+
+ case []any:
+ if items, ok := resolved["items"].(map[string]any); ok {
+ items = resolveSchemaRef(root, items)
+ for i := range v {
+ applyDefaults(root, items, v[i])
+ }
+ }
+ }
Review Comment:
`applyDefaults` only walks `properties`/`items` (and `$ref`) but does not handle schemas that use `oneOf`/`anyOf`/`allOf` (used by rag.schema.json for splitter.spec). This means schema defaults inside those branches won't be injected, and later semantic validation can fail even though the schema defines defaults. Consider extending default application to descend into composition keywords (e.g., choose a branch based on `type` discriminator when available, or apply defaults across all branches where safe).
##########
ai/component/rag/retriever.go:
##########
@@ -0,0 +1,237 @@
+package rag
+
+import (
+ "context"
+
+ "dubbo-admin-ai/utils"
+ "fmt"
+ "sync"
+
+ "github.com/cloudwego/eino/components/retriever"
+ "github.com/cloudwego/eino/schema"
+ "github.com/firebase/genkit/go/ai"
+ "github.com/firebase/genkit/go/core"
+ "github.com/firebase/genkit/go/genkit"
+ "github.com/firebase/genkit/go/plugins/localvec"
+ "github.com/firebase/genkit/go/plugins/pinecone"
+)
+
+// --- Retriever ---
+type PineconeRetriever struct {
+ g *genkit.Genkit
+ embedder string
+ target string
+ defaultK int
+ retriever map[string]ai.Retriever // keyed by target index
+}
+
+func newPineconeRetriever(g *genkit.Genkit, embedderModel string, targetIndex string, topK int) *PineconeRetriever {
+ return &PineconeRetriever{
+ g: g,
+ embedder: embedderModel,
+ target: targetIndex,
+ defaultK: topK,
+ }
+}
+
+func (r *PineconeRetriever) getRetriever(ctx context.Context, targetIndex string) (ai.Retriever, error) {
+ if targetIndex == "" {
+ targetIndex = "default"
+ }
+
+ if r.retriever == nil {
+ r.retriever = make(map[string]ai.Retriever)
+ }
+ ret := r.retriever[targetIndex]
+ if ret != nil {
Review Comment:
PineconeRetriever caches retrievers in a plain map without synchronization. `getRetriever` both reads and writes `r.retriever`, so concurrent Retrieve calls can race and panic. DevRetriever uses a mutex—PineconeRetriever should similarly guard map access (or use sync.Map).
##########
ai/component/server/server.yaml:
##########
@@ -0,0 +1,8 @@
+type: server
+spec:
+ port: 8880 # Server port
+ host: "localhost" # Server host
+ debug: false # Debug mode
Review Comment:
server.yaml sets `host: "localhost"`, which binds only to loopback and can break container/K8s deployments where the service must listen on all interfaces. Consider using `0.0.0.0` (matching the schema default) unless there’s a specific reason to restrict binding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [PR] refactor(ai): componentize runtime and configuration with schema-validated loading [dubbo-admin]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #1421:
URL: https://github.com/apache/dubbo-admin/pull/1421#issuecomment-4005115067
## [](https://sonarcloud.io/dashboard?id=apache_dubbo-admin&pullRequest=1421) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_dubbo-admin&pullRequest=1421&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_dubbo-admin&pullRequest=1421&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_dubbo-admin&pullRequest=1421&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_dubbo-admin&pullRequest=1421&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_dubbo-admin&pullRequest=1421&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_dubbo-admin&pullRequest=1421)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [PR] refactor(ai): componentize runtime and configuration with schema-validated loading [dubbo-admin]
Posted by "robocanic (via GitHub)" <gi...@apache.org>.
robocanic merged PR #1421:
URL: https://github.com/apache/dubbo-admin/pull/1421
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org