[jira] [Resolved] (ARROW-13982) [C++] Async scanner stalls if a fragment generates no batches

["David Li (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Li resolved ARROW-13982.
------------------------------
    Resolution: Fixed

Issue resolved by pull request 11145
[https://github.com/apache/arrow/pull/11145]

> [C++] Async scanner stalls if a fragment generates no batches
> -------------------------------------------------------------
>
>                 Key: ARROW-13982
>                 URL: https://issues.apache.org/jira/browse/ARROW-13982
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++
>    Affects Versions: 5.0.0
>         Environment: ubuntu 18.04 LTS
>            Reporter: Huxley Hu
>            Assignee: David Li
>            Priority: Critical
>              Labels: pull-request-available, query-engine
>             Fix For: 6.0.0
>
>         Attachments: repro.py
>
>          Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Reading parquet files using dataset scanner may stall due to a never-finished future. 
> To reproduce this case, one needs two parquet files and sets the filter expression to something that could filter one file completely.  After that, calling `AsyncScanner::ToRecordBatchReader` and read data continually. 
> I also have dug this bug a little. It's caused by the `MakeEmptyGenerator<std::shared_ptr<RecordBatch>>` when filtered row groups is empty, which's ignored by `FragmentToBatches` and causes SequencingGenerator to stall.
> A quick fix is to return a record batch with 0 rows instead of returning a nullptr there.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)