You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Jayjeet Chakraborty (Jira)" <ji...@apache.org> on 2021/06/24 13:58:00 UTC
[jira] [Commented] (ARROW-13161) [C++][Dataset] Allow setting
FragmentReadahead to 0 in ScannerBuilder
[ https://issues.apache.org/jira/browse/ARROW-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368872#comment-17368872 ]
Jayjeet Chakraborty commented on ARROW-13161:
---------------------------------------------
I see. Looks like fragment readahead has to be min. 1 for scanning to start. Also, I looked into the batch readahead parameter. Our fragments (files) have single record batches. So, it is not of much use to us at this point.
> [C++][Dataset] Allow setting FragmentReadahead to 0 in ScannerBuilder
> ---------------------------------------------------------------------
>
> Key: ARROW-13161
> URL: https://issues.apache.org/jira/browse/ARROW-13161
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Jayjeet Chakraborty
> Priority: Major
> Labels: dataset, datasets
>
> I have an application where I need to set fragment readahead to 0. But, looks like for some reason the ScannerBuilder does not allow setting the fragment readahead to 0 [1]. It would be very helpful to know why it is that way and if a PR lifting that restriction would be accepted because a docstring mentions that users can set fragment readahead to 0 if they want [2].
> [1]https://github.com/apache/arrow/blob/master/cpp/src/arrow/dataset/scanner.cc#L864
> [2]https://github.com/apache/arrow/blob/998a2a1668ea57a49d85fbb38f7f0e7eb94c29db/cpp/src/arrow/dataset/scanner.h#L93
--
This message was sent by Atlassian Jira
(v8.3.4#803005)