You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Rui Wang (Jira)" <ji...@apache.org> on 2022/12/01 18:51:00 UTC

[jira] [Assigned] (SPARK-40586) Decouple plan transformation and validation on server side

     [ https://issues.apache.org/jira/browse/SPARK-40586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rui Wang reassigned SPARK-40586:
--------------------------------

    Assignee:     (was: Rui Wang)

> Decouple plan transformation and validation on server side 
> -----------------------------------------------------------
>
>                 Key: SPARK-40586
>                 URL: https://issues.apache.org/jira/browse/SPARK-40586
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Connect
>    Affects Versions: 3.4.0
>            Reporter: Rui Wang
>            Priority: Major
>
> Project connect, from some perspectives, can be thought as replacing the SQL parser to generate a parsed (but the difference that is unresolved) plan, then the plan is passed to the analyzer. This means that connect should also do validation on the proto as there are many in-validate parser cases that analyzer does not expect to see, which potentially could cause problems if connect only pass through the proto (of course have it translated) to analyzer.
> Meanwhile I think this is a good idea to decouple the validation and transformation so that we have two stages:
> stage 1: proto validation. For example validate if necessary fields are populated or not.
> stage 2: transformation, which convert the proto to a plan with assumption that the plan is valid parsed version of the plan.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org