You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "JingsongLi (via GitHub)" <gi...@apache.org> on 2026/07/01 05:17:50 UTC

Re: [PR] [global-index-eslib] integrates the Elasticsearch (Lucene) index engine into Paimon's global index system [paimon]

JingsongLi commented on code in PR #8000:
URL: https://github.com/apache/paimon/pull/8000#discussion_r3503442235


##########
paimon-core/src/main/java/org/apache/paimon/table/source/FullTextScanImpl.java:
##########
@@ -101,16 +101,20 @@ public Plan scan() {
                         .map(IndexManifestEntry::indexFile)
                         .collect(Collectors.toList());
 
-        // Group full-text index files by column and row range.
+        // Group full-text index files by column and row range. A multi-column index serves a text
+        // column through either its primary indexFieldId or its extraFieldIds, and one file may
+        // cover more than one searched text column.
         Map<String, Map<Range, List<IndexFileMeta>>> byColumnAndRange = new HashMap<>();
         for (IndexFileMeta indexFile : allIndexFiles) {
             GlobalIndexMeta meta = checkNotNull(indexFile.globalIndexMeta());
-            String columnName = checkNotNull(idToColumn.get(meta.indexFieldId()));
             Range range = new Range(meta.rowRangeStart(), meta.rowRangeEnd());
-            byColumnAndRange
-                    .computeIfAbsent(columnName, k -> new HashMap<>())
-                    .computeIfAbsent(range, k -> new ArrayList<>())
-                    .add(indexFile);
+            for (int columnId : matchedTextColumnIds(meta, textColumnIds)) {
+                String columnName = checkNotNull(idToColumn.get(columnId));
+                byColumnAndRange
+                        .computeIfAbsent(columnName, k -> new HashMap<>())
+                        .computeIfAbsent(range, k -> new ArrayList<>())
+                        .add(indexFile);

Review Comment:
   This grouping can put multiple unrelated full-text-capable index definitions into the same split when they cover the same text column and row range. For example, a table can have a normal full-text index whose primary field is `content` and a hybrid/vector ES index whose primary field is another column but has `content` in `extraFieldIds`; both match `matchedTextColumnIds(...)` and end up in the same `FullTextSearchSplit`. `FullTextReadImpl` then picks the first file index type/meta and passes the whole list to one backend reader, but `ESIndexGlobalIndexReader` still requires `files.size() == 1` and will fail for this valid layout. Please keep separate splits/readers per index identity (for example index type + primary field + extra field set) or otherwise avoid merging files from different global index definitions into one reader input.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org