You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "CrownChu (via GitHub)" <gi...@apache.org> on 2026/07/14 07:49:41 UTC

[PR] [global-index-eslib] Fix query compatibility and vector metrics, and use thin packaging [paimon]

CrownChu opened a new pull request, #8618:
URL: https://github.com/apache/paimon/pull/8618

   ## Purpose
   
     This is a follow-up to #8000. It fixes persisted full-text query compatibility and vector metric handling, and changes `paimon-eslib` to a thin JAR so that ESLib and Lucene classes are no longer bundled into the Apache artifact.
   
     ## Changes
   
     ### Thin packaging
   
     - Remove the module-specific shading configuration that bundled ESLib and Lucene classes.
     - Keep ESLib and Lucene as regular Maven dependencies.
     - Resolve ESLib 1.0.6 artifacts from the public GitHub Maven repository.
     - Document the runtime dependency requirements in `paimon-eslib/README.md`.
     - Include `paimon-eslib` in the JDK 17 snapshot build and deployment workflow.
   
     This change only separates the artifact boundary. It does not relicense ESLib or change its dependency license.
   
     ### Persisted full-text query compatibility
   
     - Accept the serialized `column` field as a routing hint when reading persisted SQL full-text predicates.
     - Continue using the resolved physical index field for query execution.
     - Add regression coverage for persisted predicates on complementary text fields.
   
     ### Vector metric correctness
   
     - Implement the `VectorGlobalIndexer` metric contract.
     - Support canonical `l2`, `cosine`, and `inner_product` metric mappings.
     - Use the metric persisted in each index shard instead of relying on the current table configuration.
     - Reject searches across shards with inconsistent vector metrics.
     - Convert Lucene vector scores to Paimon's expected score semantics.
     - Close the reader when vector metric registration fails to avoid leaking resources.
   
     ## Tests
   
     ```shell
     mvn -pl paimon-eslib clean verify
   
     Result:
   
     Tests run: 70, Failures: 0, Errors: 0, Skipped: 0
     BUILD SUCCESS
   
     The generated paimon-eslib JAR was also verified not to contain ESLib or Lucene classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [global-index-eslib]use thin packaging and fix pom bundle,fix query compatibility and vector metrics, and [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #8618:
URL: https://github.com/apache/paimon/pull/8618


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [global-index-eslib]use thin packaging and fix pom bundle,fix query compatibility and vector metrics, and [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8618:
URL: https://github.com/apache/paimon/pull/8618#issuecomment-4966950737

   +1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org