[jira] [Updated] (HBASE-22582) The Compaction writer may access the lastCell whose memory has been released when appending fileInfo in the final

["Zheng Hu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-22582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zheng Hu updated HBASE-22582:
-----------------------------
    Component/s: Compaction

> The Compaction writer may access the lastCell whose memory has been released when appending fileInfo in the final
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-22582
>                 URL: https://issues.apache.org/jira/browse/HBASE-22582
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction
>            Reporter: Zheng Hu
>            Priority: Major
>
> Copy the comment from [~javaman_chen] under HBASE-21879: https://issues.apache.org/jira/browse/HBASE-21879?focusedCommentId=16862244&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16862244
> In Compactor#compact, we have the following:
> {code}
> protected List<Path> compact(final CompactionRequest request...
>   ...
>   try {
>     ...
>   } finally {
>     Closeables.close(scanner, true);
>     if (!finished && writer != null) {
>       abortWriter(writer);
>     }
>   }
>   assert finished : "We should have exited the method on all error paths";
>   assert writer != null : "Writer should be non-null if no error";
>   return commitWriter(writer, fd, request);
> }
> {code}
> should we call writer#beforeShipped() before Closeables.close(scanner, true);
> In order to copy some cell's data out of the ByteBuff before it released, or commitWriter may be wrong in the following call stack
> {code}
> Compactor#commitWriter
>     －> HFileWriterImpl#close
>          －> HFileWriterImpl#writeFileInfo
>                －> HFileWriterImpl#finishFileInfo
> {code}
> {code}
> protected void finishFileInfo() throws IOException {
>   if (lastCell != null) {
>     // Make a copy. The copy is stuffed into our fileinfo map. Needs a clean
>     // byte buffer. Won't take a tuple.
>     byte [] lastKey = PrivateCellUtil.getCellKeySerializedAsKeyValueKey(this.lastCell);
>     fileInfo.append(FileInfo.LASTKEY, lastKey, false);
>   }
>   ...
> }
> {code}
> Because the lastCell may refer to a reused ByteBuff. 
> Checked the code, It's a bug and will need to fix in all 2.x & master branch. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)