[jira] [Resolved] (HIVE-24530) Potential NPE in FileSinkOperator.closeRecordwriters method

["Marta Kuczora (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marta Kuczora resolved HIVE-24530.
----------------------------------
    Fix Version/s: 4.0.0
       Resolution: Fixed

Pushed to master.
Thanks a lot [~szita] for the review!

> Potential NPE in FileSinkOperator.closeRecordwriters method
> -----------------------------------------------------------
>
>                 Key: HIVE-24530
>                 URL: https://issues.apache.org/jira/browse/HIVE-24530
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 4.0.0
>            Reporter: Marta Kuczora
>            Assignee: Marta Kuczora
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> During testing a NPE occurred in the FileSinkOperator.closeRecordwriters method.
> After investigating, turned out there was an underlaying IOException during executing the FileSinkOperator.process method. It got caught by the following code part:
> {noformat}
>     } catch (IOException e) {
>       closeWriters(true);
>       throw new HiveException(e);
>     } catch (SerDeException e) {
>       closeWriters(true);
>       throw new HiveException(e);
>     }
> {noformat}
> First the closeWriters method was called:
> {noformat}
>   private void closeWriters(boolean abort) throws HiveException {
>     fpaths.closeWriters(true);
>     closeRecordwriters(true);
>   }
>   private void closeRecordwriters(boolean abort) {
>     for (RecordWriter writer : rowOutWriters) {
>       try {
>         LOG.info("Closing {} on exception", writer);
>         writer.close(abort);
>       } catch (IOException e) {
>         LOG.error("Error closing rowOutWriter" + writer, e);
>       }
>     }
> {noformat}
> If the writers had got closed successfully, a HiveException would have been thrown with the original IOException.
> But when the IOException occurred the writers in the rowOutWriters were not yet initialised, so a NPE occurred. This was very misleading as the NPE was not the real issue, but the original IOException was hidden.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)