[jira] [Comment Edited] (ARROW-11432) [Rust][DataFusion] Join Statement: Schema contains duplicate unqualified field name

["R J (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-11432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287821#comment-17287821 ] 

R J edited comment on ARROW-11432 at 2/20/21, 11:59 PM:
--------------------------------------------------------

I'm not sure there is an easy fix without making breaking changes to the public API. When building a join schema, it checks if the join set is valid (physical_plan::hash_utils::check_join_set_is_valid), which has a parent public API call (physical_plan::hash_utils::check_join_is_valid). This join is unaware of the registered name (CSV or parquet) as it is performed with arrow schemas rather than DataFusion schemas.

 

EDIT:

It could be my lack of knowledge of the DataFusion codebase, but it appears it would need a lot of changes.


was (Author: turnofacard):
I'm not sure there is an easy fix without making breaking changes to the public API. When building a join schema, it checks if the join set is valid (physical_plan::hash_utils::check_join_set_is_valid), which has a parent public API call (physical_plan::hash_utils::check_join_is_valid). This join is unaware of the registered name (CSV or parquet) as it is performed with arrow schemas rather than DataFusion schemas.

> [Rust][DataFusion] Join Statement: Schema contains duplicate unqualified field name
> -----------------------------------------------------------------------------------
>
>                 Key: ARROW-11432
>                 URL: https://issues.apache.org/jira/browse/ARROW-11432
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Rust - DataFusion
>    Affects Versions: 3.0.0
>            Reporter: GANG LIAO
>            Priority: Critical
>
> https://github.com/apache/arrow/issues/9307



--
This message was sent by Atlassian Jira
(v8.3.4#803005)