You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "anandkrshaw (via GitHub)" <gi...@apache.org> on 2025/02/07 09:06:29 UTC
[I] Reload job keep on running even after success : Pinot 1.2 [pinot]
anandkrshaw opened a new issue, #15011:
URL: https://github.com/apache/pinot/issues/15011
{
"metadata": {
"jobId": "73395ad1-453e-4a0d-9bd4-8e6a45bd0d0f",
"messageCount": "8",
"submissionTimeMs": "1727096024859",
"jobType": "RELOAD_SEGMENT",
"tableName": "amxordersntt_REALTIME"
},
"successCount": 30470,
"totalSegmentCount": 30470,
"totalServersQueried": 6,
"totalServerCallsFailed": 0,
"timeElapsedInMinutes": 197049.97985,
"estimatedTimeRemainingInMinutes": 0
}
<img width="912" alt="Image" src="https://github.com/user-attachments/assets/bb032415-a6fb-451c-b888-61322d36ea8f" />
<img width="912" alt="Image" src="https://github.com/user-attachments/assets/85066372-41a5-4705-a1b7-f913318e9b23" />
New config changes for data retention days not getting applied - Most likely because of this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Reload job keep on running even after success : Pinot 1.2 [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on issue #15011:
URL: https://github.com/apache/pinot/issues/15011#issuecomment-2647811694
https://github.com/apache/pinot/issues/15022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Reload job keep on running even after success : Pinot 1.2 [pinot]
Posted by "anandkrshaw (via GitHub)" <gi...@apache.org>.
anandkrshaw closed issue #15011: Reload job keep on running even after success : Pinot 1.2
URL: https://github.com/apache/pinot/issues/15011
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Reload job keep on running even after success : Pinot 1.2 [pinot]
Posted by "anandheritage (via GitHub)" <gi...@apache.org>.
anandheritage commented on issue #15011:
URL: https://github.com/apache/pinot/issues/15011#issuecomment-2642609796
Hi Team , For one of our table the older segments are not getting deleted. The TTL is of 30 days but the data is still there for beyond (100 days).
The table ingestion is going good and able to serve queries correctly too. Segments status is good too for All.
Although there are few observations :
private void manageRetentionForRealtimeTable(String realtimeTableName, RetentionStrategy retentionStrategy) {
List<String> segmentsToDelete = new ArrayList<>();
IdealState idealState = _pinotHelixResourceManager.getHelixAdmin()
.getResourceIdealState(_pinotHelixResourceManager.getHelixClusterName(), realtimeTableName);
for (SegmentZKMetadata segmentZKMetadata : _pinotHelixResourceManager.getSegmentsZKMetadata(realtimeTableName)) {
String segmentName = segmentZKMetadata.getSegmentName();
if (segmentZKMetadata.getStatus() == [Status.IN](http://status.in/)_PROGRESS) {
// Delete old LLC segment that hangs around. Do not delete segment that are current since there may be a race
// with RealtimeSegmentValidationManager trying to auto-create the LLC segment
if (shouldDeleteInProgressLLCSegment(segmentName, idealState, segmentZKMetadata)) {
segmentsToDelete.add(segmentName);
}
} else {
// Sealed segment
if (retentionStrategy.isPurgeable(realtimeTableName, segmentZKMetadata)) {
segmentsToDelete.add(segmentName);
}
}
}
_pinotHelixResourceManager.getSegmentsZKMetadata(realtimeTableName))
For some reason the segment path is not there under segments.
But I can see the metadata under External View.
For all the other tables I can see ZK metadata is present in both the places.
Controller logs : 2025-02-07T09:56:52.630692061Z stdout F ZKPath /PinotCluster/PROPERTYSTORE/SEGMENTS/amxordersntt_REALTIME/533/code/code/code/code/code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Reload job keep on running even after success : Pinot 1.2 [pinot]
Posted by "anandheritage (via GitHub)" <gi...@apache.org>.
anandheritage commented on issue #15011:
URL: https://github.com/apache/pinot/issues/15011#issuecomment-2642610369
[
](url)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org