[GitHub] [arrow-datafusion] mustafasrepo opened a new issue, #6510: SELECTing 0 column with EXCLUDE gives projection_push_down error

["mustafasrepo (via GitHub)" <gi...@apache.org>]

mustafasrepo opened a new issue, #6510:
URL: https://github.com/apache/arrow-datafusion/issues/6510

   ### Describe the bug
   
   Assume we have a table with single column `ts` such as below
   |a|
   |----|
   |1|
   |2|
   |3|
   |4|
   when I run the query below on this table
   ```sql
   SELECT * EXCEPT(a)
   FROM table1
   ```
   It gives `Error: Context("push_down_projection", Internal("Optimizer rule 'push_down_projection' failed, due to generate a different schema, original schema: DFSchema { fields: [], metadata: {} }, new schema: DFSchema { fields: [DFField { qualifier: Some(Bare { table: \"table1\" }), field: Field { name: \"a\", data_type: Int32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} } }], metadata: {} }"))` error.
   
   However, assume we have an empty table with no columns named `empty_table`. Query below works
   ```sql
   SELECT *
   FROM empty_table
   ```
   So I don't think bug is related to showing empty columns, but I am not sure.
   
   
   
   ### To Reproduce
   
   You can use test below to reproduce problem
   ```rust
   #[tokio::test]
   async fn test_exclude_error() -> Result<()> {
       let config = SessionConfig::new()
           .with_target_partitions(1);
       let ctx = SessionContext::with_config(config);
       ctx.sql("CREATE TABLE table1 (
             a INT,
           ) as VALUES
                 (1),
                 (2),
                 (3),
                 (4)").await?;
   
       let sql = "SELECT * EXCEPT(a)
                       FROM table1";
   
       let msg = format!("Creating logical plan for '{sql}'");
       let dataframe = ctx.sql(sql).await.expect(&msg);
       let physical_plan = dataframe.create_physical_plan().await?;
       let batches = collect(physical_plan, ctx.task_ctx()).await?;
       print_batches(&batches)?;
       Ok(())
   }
   ```
   
   Test below shows that datafusion has no problem with showing 0 columns.
   ```rust
   #[tokio::test]
   async fn test_empty_table() -> Result<()> {
       let config = SessionConfig::new()
           .with_target_partitions(1);
       let ctx = SessionContext::with_config(config);
       ctx.sql("CREATE TABLE empty_table ()").await?;
   
       let sql = "SELECT *
                       FROM empty_table";
   
       let msg = format!("Creating logical plan for '{sql}'");
       let dataframe = ctx.sql(sql).await.expect(&msg);
       let physical_plan = dataframe.create_physical_plan().await?;
       let batches = collect(physical_plan, ctx.task_ctx()).await?;
       print_batches(&batches)?;
       Ok(())
   }
   ```
   
   ### Expected behavior
   
   I expect first query to run successfully
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] SELECTing 0 column with EXCLUDE gives projection_push_down error [arrow-datafusion]

["Jefffrey (via GitHub)" <gi...@apache.org>]

Jefffrey commented on issue #6510:
URL: https://github.com/apache/arrow-datafusion/issues/6510#issuecomment-1782524357

   Issue looks resolved as of current main:
   
   ```sql
   datafusion-cli$ cargo run
       Finished dev [unoptimized + debuginfo] target(s) in 0.28s
        Running `/media/jeffrey/1tb_860evo_ssd/.cargo_target_cache/debug/datafusion-cli`
   DataFusion CLI v32.0.0
   ❯ CREATE TABLE table1 (a INT) as VALUES (1), (2), (3), (4);
   0 rows in set. Query took 0.004 seconds.
   
   ❯ SELECT * EXCEPT(a) FROM table1;
   ++
   ++
   ++
   4 rows in set. Query took 0.002 seconds.
   
   ❯
   ```
   
   - to note it still returns 4 rows, just in an empty schema
   
   I've added a test to sqllogictest to ensure this will be in test suite: https://github.com/apache/arrow-datafusion/pull/7945


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] SELECTing 0 column with EXCLUDE gives projection_push_down error [arrow-datafusion]

["alamb (via GitHub)" <gi...@apache.org>]

alamb closed issue #6510: SELECTing 0 column with EXCLUDE gives projection_push_down error
URL: https://github.com/apache/arrow-datafusion/issues/6510


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org