[jira] [Created] (SPARK-35421) Remove redundant ProjectExec from streaming queries with V2Relation

["Kousuke Saruta (Jira)" <ji...@apache.org>]

Kousuke Saruta created SPARK-35421:
--------------------------------------

             Summary: Remove redundant ProjectExec from streaming queries with V2Relation
                 Key: SPARK-35421
                 URL: https://issues.apache.org/jira/browse/SPARK-35421
             Project: Spark
          Issue Type: Improvement
          Components: Structured Streaming
    Affects Versions: 3.2.0
            Reporter: Kousuke Saruta
            Assignee: Kousuke Saruta


Streaming queries with V2Relation can have redundant ProjectExec in it's physical plan.
You can easily reproduce with the following code.

{code}
import org.apache.spark.sql.streaming.Trigger

val query = spark.
  readStream.
  format("rate").
  option("rowsPerSecond", 1000).
  option("rampUpTime", "10s").
  load().
  selectExpr("timestamp", "100",  "value").  
  writeStream.
  format("console").
  trigger(Trigger.ProcessingTime("5 seconds")).
  // trigger(Trigger.Continuous("5 seconds")). // You can reproduce with continuous processing too.
  outputMode("append").
  start()
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org