You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "airborne12 (via GitHub)" <gi...@apache.org> on 2026/10/23 07:35:58 UTC
[PR] branch-2.0: [refactor](clucene) Replace the GPL TurboPFor library with an independent PFOR codec (#421) [doris-thirdparty]
airborne12 opened a new pull request, #425:
URL: https://github.com/apache/doris-thirdparty/pull/425
### What problem does this PR solve?
Issue Number: None
Related PR: #421
Problem Summary:
Pick of #421 to clucene-2.0. `src/ext/for` is a vendored copy of powturbo's TurboPFor, which is licensed under GPL v2, linked into clucene-core and thereby into every Doris BE. It is replaced by `CLucene/util/PForCodec.{h,cpp}`, an independent implementation of the same on-disk block format (bit compatible in both directions), and `src/ext/for`, the `ic` library target and the `ic` link dependencies are removed. See #421 for the details, the measurements and the tests of the codec.
Differences to the original change: the pfor functions of this branch are the free functions `lucene::util::pfor_encode / pfor_decode` and there is no `PforUtil::encodePos / decodePos`. The branch keeps its choice of the format by platform (including that kPfor128 / kPfor256 are rejected where they were), only the TurboPFor calls are replaced by the new codec. The tests are adapted to these names and the tests of encodePos / decodePos are left out. The root CMakeLists.txt keeps its own `src/ext/sse2neon` include directory.
### Release note
Removed the GPL v2 licensed TurboPFor library from the inverted index code. The index format is unchanged.
### Check List (For Author)
- Test: Unit Test
- Full cl_test (63 tests) built with clang + AVX2 (with `-include memory`, see above) and with g++-11 and the default options: all passed.
- Behavior changed: No
- Does this need documentation: No
(cherry picked from commit 5725b460b6b12a6874b171832f9f834ce161b2f4)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-2.0: [refactor](clucene) Replace the GPL TurboPFor library with an independent PFOR codec (#421) [doris-thirdparty]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 merged PR #425:
URL: https://github.com/apache/doris-thirdparty/pull/425
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org