You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Keston375 (via GitHub)" <gi...@apache.org> on 2025/01/22 20:22:32 UTC
[I] Bug Report Support Upsert deletion for TTL: construct queryableDocIds when adding segments out of TTL [pinot]
Keston375 opened a new issue, #14885:
URL: https://github.com/apache/pinot/issues/14885
# Title
Bugfix: Construct `queryableDocIds` When Adding Segments Out of TTL in Upsert Tables
## Repository Version
release-1.0.0
## Key Words / Search Terms
upsert, TTL, `queryableDocIds`, segment addition, deletion, `upsertConfig`, `deleteRecordColumn`
## Description
In Apache Pinot, upsert tables support both soft-deletes via a designated boolean `deleteRecordColumn` and a time-to-live (TTL) mechanism to manage the lifecycle of primary keys. Previously, when adding segments that fall outside the defined TTL, the system did not construct the `queryableDocIds`, leading to potential inconsistencies in query results. This bugfix addresses the issue by ensuring that `queryableDocIds` are properly constructed when segments beyond the TTL are added, thereby maintaining accurate and consistent query responses.
## Expected Behavior
- When segments that exceed the defined TTL are added to an upsert table, the system should construct the `queryableDocIds` to reflect the current state of data accurately.
- Queries executed against the upsert table should return results consistent with the applied deletions and TTL configurations.
## Current Behavior
- Segments added beyond the TTL do not trigger the construction of `queryableDocIds`, resulting in potential discrepancies in query outcomes.
- Soft-deleted records may still appear in query results due to the absence of updated `queryableDocIds`.
## Use Cases / Motivating Examples (for Feature Requests)
- Ensuring that upsert tables with both deletion and TTL configurations maintain data integrity and provide accurate query results.
- Preventing scenarios where outdated or soft-deleted records are inadvertently included in query responses due to missing `queryableDocIds`.
- Enhancing the reliability of upsert tables in handling complex data lifecycle management involving both deletions and TTL policies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Bug Report Support Upsert deletion for TTL: construct queryableDocIds when adding segments out of TTL [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #14885: Bug Report Support Upsert deletion for TTL: construct queryableDocIds when adding segments out of TTL
URL: https://github.com/apache/pinot/issues/14885
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Bug Report Support Upsert deletion for TTL: construct queryableDocIds when adding segments out of TTL [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #14885:
URL: https://github.com/apache/pinot/issues/14885#issuecomment-2608586769
Suspect this is AI generated. Closing it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Bug Report Support Upsert deletion for TTL: construct queryableDocIds when adding segments out of TTL [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #14885:
URL: https://github.com/apache/pinot/issues/14885#issuecomment-2608567827
cc @tibrewalpratik17
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org