[jira] [Updated] (FLINK-5518) HadoopInputFormat throws NPE when close() is called before open()

["Jakub Havlik (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-5518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jakub Havlik updated FLINK-5518:
--------------------------------
    Description: 
When developing a simple Flink applications reading ORC files it crashes with NullPointerException when number of instances/executor threads is higher then the number of files because it is trying to close a HadoopInputFormat which is trying to close RecordReader which was not yet initialized as there is no file for which it should have been opened. The issue is caused when
{code:java}
public void run(SourceContext<OUT> ctx) throws Exception {
    try {
...
        while (isRunning) {
	    format.open(splitIterator.next());
...
    } finally {
	format.close();
...
    }
{code}
in file {{InputFormatSourceFunction.java}} which calls
{code:java}
public void close() throws IOException {
    // enforce sequential close() calls
    synchronized (CLOSE_MUTEX) {
        this.recordReader.close();
    }
}
{code}
from {{HadoopInputFormatBase.java}}.

As there is just this one implementation of the {{close()}} method it may be enough just to add a null check for the {{this.recordReader}} in there.

  was:
When developing a simple Flink applications reading ORC files it crashes with NullPointerException when number of instances/executor threads is higher then the number of files because it is trying to close a HadoopInputFormat which is trying to close RecordReader which was not yet initialized as there is no file for which it should have been opened. The issue is caused when
{code:java}
public void run(SourceContext<OUT> ctx) throws Exception {
    try {
...
        while (isRunning) {
	    format.open(splitIterator.next());
...
    } finally {
	format.close();
...
    }
{code}
in file {{InputFormatSourceFunction.java}} which calls
{code:java}
public void close() throws IOException {

		// enforce sequential close() calls
		synchronized (CLOSE_MUTEX) {
			this.recordReader.close();
		}
	}
{code}
from {{HadoopInputFormatBase.java}}.

As there is just this one implementation of the {{close()}} method it may be enough just to add a null check for the {{this.recordReader}} in there.


> HadoopInputFormat throws NPE when close() is called before open()
> -----------------------------------------------------------------
>
>                 Key: FLINK-5518
>                 URL: https://issues.apache.org/jira/browse/FLINK-5518
>             Project: Flink
>          Issue Type: Bug
>          Components: Batch Connectors and Input/Output Formats
>    Affects Versions: 1.1.4
>            Reporter: Jakub Havlik
>              Labels: beginner, easyfix, newbie
>
> When developing a simple Flink applications reading ORC files it crashes with NullPointerException when number of instances/executor threads is higher then the number of files because it is trying to close a HadoopInputFormat which is trying to close RecordReader which was not yet initialized as there is no file for which it should have been opened. The issue is caused when
> {code:java}
> public void run(SourceContext<OUT> ctx) throws Exception {
>     try {
> ...
>         while (isRunning) {
> 	    format.open(splitIterator.next());
> ...
>     } finally {
> 	format.close();
> ...
>     }
> {code}
> in file {{InputFormatSourceFunction.java}} which calls
> {code:java}
> public void close() throws IOException {
>     // enforce sequential close() calls
>     synchronized (CLOSE_MUTEX) {
>         this.recordReader.close();
>     }
> }
> {code}
> from {{HadoopInputFormatBase.java}}.
> As there is just this one implementation of the {{close()}} method it may be enough just to add a null check for the {{this.recordReader}} in there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)