You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by "Chaitanya (JIRA)" <ji...@apache.org> on 2017/03/14 07:22:41 UTC
[jira] [Updated] (APEXMALHAR-2407) Time buckets are not purging
after expiry
[ https://issues.apache.org/jira/browse/APEXMALHAR-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chaitanya updated APEXMALHAR-2407:
----------------------------------
Description:
Below are the observations:
1) purgeListner is not triggering purgeTimeBucketsLessThanEqualTo() in MovingTimeBucketAssigner.
This will trigger only if the value of "triggerPurge" is true.
"triggerPurge" calculated based on (diffInBuckets > 0) condition. "diffInBuckets" is calculated based on below condition:
if (time >= end)
{ long diffInBuckets = (time - end) / bucketSpanMillis; ------ }
If the data is processing time then the value of diffInBuckets is always zero.
2) Comparing the key bucket Id with time bucket while writing bucketed data which was happening in transferWindowFiles() which is in IncrementalCheckpointManager.
> Time buckets are not purging after expiry
> -----------------------------------------
>
> Key: APEXMALHAR-2407
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2407
> Project: Apache Apex Malhar
> Issue Type: Sub-task
> Reporter: Chaitanya
> Assignee: Chaitanya
>
> Below are the observations:
> 1) purgeListner is not triggering purgeTimeBucketsLessThanEqualTo() in MovingTimeBucketAssigner.
> This will trigger only if the value of "triggerPurge" is true.
> "triggerPurge" calculated based on (diffInBuckets > 0) condition. "diffInBuckets" is calculated based on below condition:
> if (time >= end)
> { long diffInBuckets = (time - end) / bucketSpanMillis; ------ }
> If the data is processing time then the value of diffInBuckets is always zero.
> 2) Comparing the key bucket Id with time bucket while writing bucketed data which was happening in transferWindowFiles() which is in IncrementalCheckpointManager.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)