[jira] [Commented] (APEXMALHAR-2407) Time buckets are not purging after expiry

["Bhupesh Chawda (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923999#comment-15923999 ] 

Bhupesh Chawda commented on APEXMALHAR-2407:
--------------------------------------------

Fixes included in PR: https://github.com/apache/apex-malhar/pull/551


> Time buckets are not purging after expiry
> -----------------------------------------
>
>                 Key: APEXMALHAR-2407
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2407
>             Project: Apache Apex Malhar
>          Issue Type: Sub-task
>            Reporter: Chaitanya
>            Assignee: Chaitanya
>
> Below are the observations:
> 1) purgeListner is not triggering purgeTimeBucketsLessThanEqualTo() in MovingTimeBucketAssigner.
> This will trigger only if the value of "triggerPurge" is true.
> "triggerPurge" calculated based on (diffInBuckets > 0) condition. "diffInBuckets" is calculated based on below condition:
> if (time >= end)
> { long diffInBuckets = (time - end) / bucketSpanMillis; ------ }
> If the data is processing time then the value of diffInBuckets is always zero.
> 2) Comparing the key bucket Id with time bucket while writing bucketed data which was happening in transferWindowFiles() which is in IncrementalCheckpointManager.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)