Re: [PR] Upgrade lucene to 9.8.0 and upgrade text index version [pinot]

["xiangfu0 (via GitHub)" <gi...@apache.org>]

xiangfu0 commented on PR #11857:
URL: https://github.com/apache/pinot/pull/11857#issuecomment-1791491957

   > 👍 discussed with @xiangfu0 offline
   > 
   > * the best way is to keep both versions of the index until the next release, by using different file paths. Existing segment with index wont change
   > * generate on the fly for the first time the segments with old indexes are downloaded and loaded the first time
   > * as for the cost, i dont think we need to regen the new index again unless the local file storage is gone, but please confirm
   > * as far as i know this PR doesn't address clean up or modifying deep-store segments
   
   Synced with @Jackie-Jiang offline as well.
   We tested there is a package called `lucene-backward-codecs` which has the old codec for pinot to read the index built by older lucene, so the change is similar to previous one, but more backward compatible as:
   1. for upgrade, new version will still try to read old index if exists, or build new index
   2. for downgrade, old index will be used, so no need to rebuild index.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org