You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Chirag Dewan via user <us...@flink.apache.org> on 2023/10/26 05:33:22 UTC
Invalid Null Check in DefaultFileFilter
Hi,
I was looking at this check in DefaultFileFilter:
public boolean test(Path path) {
final String fileName = path.getName();
if (fileName == null || fileName.length() == 0) {
return true;
}Was wondering how can a file name be null?
And if null, shouldnt this be return false?
I created a JIRA for this - [FLINK-33367] Invalid Check in DefaultFileFilter - ASF JIRA
|
|
| |
[FLINK-33367] Invalid Check in DefaultFileFilter - ASF JIRA
|
|
|
Any input is appreciated.
Thanks
Re: Invalid Null Check in DefaultFileFilter
Posted by Alexander Fedulov <al...@gmail.com>.
* with regards to empty string. The null check is still a bit defensive and
one could return false in test(), but it does not matter really since
String.substring in getName() can never return null.
On Fri, 27 Oct 2023 at 16:32, Alexander Fedulov <al...@gmail.com>
wrote:
> Actually, this is not even "defensive programming", but is the required
> logic for processing directories.
> See here:
>
> https://github.com/apache/flink/blob/release-1.18/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/src/enumerate/NonSplittingRecursiveEnumerator.java#L90
>
>
> https://github.com/apache/flink/blob/release-1.18/flink-core/src/main/java/org/apache/flink/core/fs/Path.java#L295
>
> Returning false would prevent addSplitsForPath from adding all nested
> files recursively.
>
> Best,
> Alexander
>
>
>
> On Fri, 27 Oct 2023 at 04:04, Chirag Dewan <ch...@yahoo.in>
> wrote:
>
>> Yeah agree, not a problem in general. But it just seems odd. Returning
>> true if a fileName can be null will blow up a lot more in the reader as far
>> as my understanding goes.
>>
>> I just want to understand whether this is an erroneous condition or an
>> actual use case. Lets say is it possible to get a null file name for some
>> sub directories and hence important to return true so that the File Source
>> can monitor inside those sub directories?
>>
>> On Friday, 27 October, 2023 at 12:58:44 am IST, Alexander Fedulov <
>> alexander.fedulov@gmail.com> wrote:
>>
>>
>> Is there an actual issue behind this question?
>>
>> In general: this is a form of defensive programming for a public
>> interface and the decision here is to be more lenient when facing
>> potentially erroneous user input rather than blow up the whole application
>> with a NullPointerException.
>>
>> Best,
>> Alexander Fedulov
>>
>> On Thu, 26 Oct 2023 at 07:35, Chirag Dewan via user <
>> user@flink.apache.org> wrote:
>>
>> Hi,
>>
>> I was looking at this check in DefaultFileFilter:
>>
>> public boolean test(Path path) {
>> final String fileName = path.getName();
>> if (fileName == null || fileName.length() == 0) {
>> return true;
>> }
>>
>> Was wondering how can a file name be null?
>>
>> And if null, shouldnt this be *return false*?
>>
>> I created a JIRA for this - [FLINK-33367] Invalid Check in
>> DefaultFileFilter - ASF JIRA
>> <https://issues.apache.org/jira/browse/FLINK-33367>
>>
>> [FLINK-33367] Invalid Check in DefaultFileFilter - ASF JIRA
>>
>> <https://issues.apache.org/jira/browse/FLINK-33367>
>> Any input is appreciated.
>>
>> Thanks
>>
>>
>>
>>
Re: Invalid Null Check in DefaultFileFilter
Posted by Alexander Fedulov <al...@gmail.com>.
Actually, this is not even "defensive programming", but is the required
logic for processing directories.
See here:
https://github.com/apache/flink/blob/release-1.18/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/src/enumerate/NonSplittingRecursiveEnumerator.java#L90
https://github.com/apache/flink/blob/release-1.18/flink-core/src/main/java/org/apache/flink/core/fs/Path.java#L295
Returning false would prevent addSplitsForPath from adding all nested files
recursively.
Best,
Alexander
On Fri, 27 Oct 2023 at 04:04, Chirag Dewan <ch...@yahoo.in> wrote:
> Yeah agree, not a problem in general. But it just seems odd. Returning
> true if a fileName can be null will blow up a lot more in the reader as far
> as my understanding goes.
>
> I just want to understand whether this is an erroneous condition or an
> actual use case. Lets say is it possible to get a null file name for some
> sub directories and hence important to return true so that the File Source
> can monitor inside those sub directories?
>
> On Friday, 27 October, 2023 at 12:58:44 am IST, Alexander Fedulov <
> alexander.fedulov@gmail.com> wrote:
>
>
> Is there an actual issue behind this question?
>
> In general: this is a form of defensive programming for a public interface
> and the decision here is to be more lenient when facing potentially
> erroneous user input rather than blow up the whole application with a
> NullPointerException.
>
> Best,
> Alexander Fedulov
>
> On Thu, 26 Oct 2023 at 07:35, Chirag Dewan via user <us...@flink.apache.org>
> wrote:
>
> Hi,
>
> I was looking at this check in DefaultFileFilter:
>
> public boolean test(Path path) {
> final String fileName = path.getName();
> if (fileName == null || fileName.length() == 0) {
> return true;
> }
>
> Was wondering how can a file name be null?
>
> And if null, shouldnt this be *return false*?
>
> I created a JIRA for this - [FLINK-33367] Invalid Check in
> DefaultFileFilter - ASF JIRA
> <https://issues.apache.org/jira/browse/FLINK-33367>
>
> [FLINK-33367] Invalid Check in DefaultFileFilter - ASF JIRA
>
> <https://issues.apache.org/jira/browse/FLINK-33367>
> Any input is appreciated.
>
> Thanks
>
>
>
>
Re: Invalid Null Check in DefaultFileFilter
Posted by Chirag Dewan via user <us...@flink.apache.org>.
Yeah agree, not a problem in general. But it just seems odd. Returning true if a fileName can be null will blow up a lot more in the reader as far as my understanding goes.
I just want to understand whether this is an erroneous condition or an actual use case. Lets say is it possible to get a null file name for some sub directories and hence important to return true so that the File Source can monitor inside those sub directories?
On Friday, 27 October, 2023 at 12:58:44 am IST, Alexander Fedulov <al...@gmail.com> wrote:
Is there an actual issue behind this question?
In general: this is a form of defensive programming for a public interface and the decision here is to be more lenient when facing potentially erroneous user input rather than blow up the whole application with a NullPointerException.
Best,Alexander Fedulov
On Thu, 26 Oct 2023 at 07:35, Chirag Dewan via user <us...@flink.apache.org> wrote:
Hi,
I was looking at this check in DefaultFileFilter:
public boolean test(Path path) {
final String fileName = path.getName();
if (fileName == null || fileName.length() == 0) {
return true;
}Was wondering how can a file name be null?
And if null, shouldnt this be return false?
I created a JIRA for this - [FLINK-33367] Invalid Check in DefaultFileFilter - ASF JIRA
|
|
| |
[FLINK-33367] Invalid Check in DefaultFileFilter - ASF JIRA
|
|
|
Any input is appreciated.
Thanks
Re: Invalid Null Check in DefaultFileFilter
Posted by Alexander Fedulov <al...@gmail.com>.
Is there an actual issue behind this question?
In general: this is a form of defensive programming for a public interface
and the decision here is to be more lenient when facing potentially
erroneous user input rather than blow up the whole application with a
NullPointerException.
Best,
Alexander Fedulov
On Thu, 26 Oct 2023 at 07:35, Chirag Dewan via user <us...@flink.apache.org>
wrote:
> Hi,
>
> I was looking at this check in DefaultFileFilter:
>
> public boolean test(Path path) {
> final String fileName = path.getName();
> if (fileName == null || fileName.length() == 0) {
> return true;
> }
>
> Was wondering how can a file name be null?
>
> And if null, shouldnt this be *return false*?
>
> I created a JIRA for this - [FLINK-33367] Invalid Check in
> DefaultFileFilter - ASF JIRA
> <https://issues.apache.org/jira/browse/FLINK-33367>
>
> [FLINK-33367] Invalid Check in DefaultFileFilter - ASF JIRA
>
> <https://issues.apache.org/jira/browse/FLINK-33367>
> Any input is appreciated.
>
> Thanks
>
>
>
>