[jira] [Commented] (KAFKA-463) log.truncateTo needs to handle targetOffset smaller than the lowest offset in the log

["Jun Rao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454947#comment-13454947 ] 

Jun Rao commented on KAFKA-463:
-------------------------------

Thanks for the patch. A few comments:

1. Log.truncateTo(): 
1.1 Should synchronize on the lock since we are touching the segment files.
1.2 It doesn't seem that we need to call segments.truncLast(truncatedSegmentIndex) since we are deleting those segments whose start is larger than targetOffset later. If so, we can get rid of of SegmentList.truncLast and the associated unit test. Also, it seems that the current logic will fail to delete some segment files (only the in memory segment list is updated) if there the startOffset is in the middle of a list of segments. 
1.3 The following piece of code is duplicated in 3 places: creating the first segment with offset 0, maybe roll and runcateTo(). Can we create a separate function (something like rollSegement(startOffset)) to share the code?
      val newFile = new File(dir, nameFromOffset(targetOffset))
      if (newFile.exists) {
        warn("newly rolled logsegment " + newFile.getName + " already exists; deleting it first")
        newFile.delete()
      }
      debug("Rolling log '" + name + "' to " + newFile.getName())
      segments.append(new LogSegment(newFile, new FileMessageSet(newFile, true), targetOffset, time))
1.4 Can we add a unit test to verify that if targetOffset is smaller that the smallest offset in the log, after tuncateTo is called, log size is 0 and logEndOffset is targetOffset?
1.5 The following message should say this is failure during truncateTo.
      error("Failed to delete some segments during log recovery")

2. FileMessageSet.truncateTo() : Condition     if(targetSize >= sizeInBytes())  should be changed to     if(targetSize > sizeInBytes())

                
> log.truncateTo needs to handle targetOffset smaller than the lowest offset in the log
> -------------------------------------------------------------------------------------
>
>                 Key: KAFKA-463
>                 URL: https://issues.apache.org/jira/browse/KAFKA-463
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>            Reporter: Jun Rao
>            Assignee: Swapnil Ghike
>            Priority: Blocker
>              Labels: bugs
>             Fix For: 0.8
>
>         Attachments: kafka-463-v1.patch
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> When this happens, we should truncate all existing log segments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira