You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Weston Pace (Jira)" <ji...@apache.org> on 2022/09/08 14:51:00 UTC
[jira] [Updated] (ARROW-17648) [C++] Add ScanOptions to support projection and filter in ToProto Read
[ https://issues.apache.org/jira/browse/ARROW-17648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weston Pace updated ARROW-17648:
--------------------------------
Description:
When converting from an Acero plan to a Substrait plan not all scan options are fully supported. The `projection` (partial column selection) and `filter` (pushdown filtering) support needs to be added.
Ref:
{code:java}
Result<std::unique_ptr<substrait::ReadRel>> ScanRelationConverter(const std::shared_ptr<Schema>& schema, const compute::Declaration& declaration, ExtensionSet* ext_set, const ConversionOptions& conversion_options) {
auto read_rel = make_unique<substrait::ReadRel>();
const auto& scan_node_options = checked_cast<const dataset::ScanNodeOptions&>(*declaration.options);
auto dataset = dynamic_cast<dataset::FileSystemDataset*>(scan_node_options.dataset.get());
// TODO handle here{code}
was:
In the current ToProto the `ScanOptions` are not fully supported, the `project` and `filter` support needs to be added.
Ref:
{code:java}
Result<std::unique_ptr<substrait::ReadRel>> ScanRelationConverter(const std::shared_ptr<Schema>& schema, const compute::Declaration& declaration, ExtensionSet* ext_set, const ConversionOptions& conversion_options) {
auto read_rel = make_unique<substrait::ReadRel>();
const auto& scan_node_options = checked_cast<const dataset::ScanNodeOptions&>(*declaration.options);
auto dataset = dynamic_cast<dataset::FileSystemDataset*>(scan_node_options.dataset.get());
// TODO handle here{code}
> [C++] Add ScanOptions to support projection and filter in ToProto Read
> ----------------------------------------------------------------------
>
> Key: ARROW-17648
> URL: https://issues.apache.org/jira/browse/ARROW-17648
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Vibhatha Lakmal Abeykoon
> Assignee: Vibhatha Lakmal Abeykoon
> Priority: Major
> Labels: query-engine, substrait, substrait-roundtrip
>
> When converting from an Acero plan to a Substrait plan not all scan options are fully supported. The `projection` (partial column selection) and `filter` (pushdown filtering) support needs to be added.
> Ref:
>
> {code:java}
> Result<std::unique_ptr<substrait::ReadRel>> ScanRelationConverter(const std::shared_ptr<Schema>& schema, const compute::Declaration& declaration, ExtensionSet* ext_set, const ConversionOptions& conversion_options) {
> auto read_rel = make_unique<substrait::ReadRel>();
> const auto& scan_node_options = checked_cast<const dataset::ScanNodeOptions&>(*declaration.options);
> auto dataset = dynamic_cast<dataset::FileSystemDataset*>(scan_node_options.dataset.get());
> // TODO handle here{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)