[GitHub] [arrow-datafusion] alamb commented on issue #1818: A corner bug in union

[GitBox <gi...@apache.org>]

alamb commented on issue #1818:
URL: https://github.com/apache/arrow-datafusion/issues/1818#issuecomment-1039599084


   Nice find @xudong963 
   
   > Currently, DF judges if the logical plans in union have the same schema by arrow schema's field name
   
   This sounds like the problem to me -- perhaps we need to add a special "does this schema match for the purposes of union" type check 🤔 
   
   
   Another thought I had would be to add `LogicalPlan::Projection` to map the names of union inputs to be compatible (assuming that the types are the same). Something like this perhaps:
   
   Plan now;
   ```text
   Union --> results in error because `x` not the same as `a` and `y` not the same as `b`
       ... original input for (x,y)
       ... original input  for (a, b)
   ```
   
   use a `Projection` to remap the names:
   
   ```text
   Union
     Projection(x as "column1", y as "column2")
       ... original input for (x,y)
     Projection(a as "column1", b as "column2")
       ... original input  for (a, b)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org