You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Phabricator (JIRA)" <ji...@apache.org> on 2012/12/16 00:08:12 UTC

[jira] [Updated] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor

     [ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Phabricator updated HIVE-3809:
------------------------------

    Attachment: D7419.1.patch

mbautin requested code review of "[jira] [HIVE-3809] Concurrency issue in RCFile: multiple threads can use the same decompressor".
Reviewers: kevinwilfong, njain, ashutoshc, cwsteinbach, edwardcapriolo, JIRA

  Making sure that decompressors are only returned to the pool once when readers are closed in RCFile. Otherwise the same decompressor can end up in the pool multiple times, and multiple threads would pick it up and try to use it concurrently.

TEST PLAN
  Unit tests

REVISION DETAIL
  https://reviews.facebook.net/D7419

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/17709/

To: kevinwilfong, njain, ashutoshc, cwsteinbach, edwardcapriolo, JIRA, mbautin

                
> Concurrency issue in RCFile: multiple threads can use the same decompressor
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-3809
>                 URL: https://issues.apache.org/jira/browse/HIVE-3809
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Mikhail Bautin
>            Priority: Critical
>         Attachments: D7419.1.patch
>
>
> RCFile is not thread-safe, even if each reader is only used by one thread as intended, because it is possible to return decompressors to the pool multiple times by calling close on the reader multiple times. Then, different threads can pick up the same decompressor twice from the pool, resulting in decompression failures.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira