[I] [Bug] A nondeterministic permanent view is recomputed for every CTE reference under authz [kyuubi]

["j1wonpark (via GitHub)" <gi...@apache.org>]

j1wonpark opened a new issue, #7746:
URL: https://github.com/apache/kyuubi/issues/7746

   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   
   ### Search before asking
   
   - [x] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
   
   ### Describe the bug
   
   With the authz extension enabled, a permanent view over a nondeterministic expression is
   recomputed for every reference when it is referenced more than once through a CTE. Without the
   extension Spark computes it once and shares the result through a `ReusedExchange`.
   
   ```sql
   CREATE VIEW v AS SELECT id, reflect('java.util.UUID', 'randomUUID') AS r FROM range(20);
   WITH c AS (SELECT * FROM v)
   SELECT a.id, a.r, b.r FROM c a JOIN c b ON a.id = b.id;
   -- authz enabled: a.r <> b.r on every row; authz disabled: a.r = b.r
   ```
   
   `InlineCTE.shouldInline` inlines a CTE that is referenced more than once only when
   `cteDef.deterministic` holds. `PermanentViewMarker` is a `LeafNode`, so `QueryPlan.deterministic`
   sees no expressions and no children and reports `true` for any view, including one over
   `rand()`, `uuid()` or `reflect()`. The `Inline CTE` batch runs before the extended optimizer
   batch that removes the marker, so the wrong answer is what `InlineCTE` sees.
   
   For seeded expressions such as `uuid()` and `rand()` the inlined copies happen to agree because
   both copies share the seed and the partition layout, so the visible symptom is the duplicated
   work. For anything that draws a fresh value per evaluation the two references disagree.
   
   Reproduced on Spark 3.5, 4.0, 4.1 and 4.2 (the marker has been a `LeafNode` since #5780, released in 1.9.0).
   
   ### Affects Version(s)
   
   master, 1.12.0, 1.11.x, 1.10.x, 1.9.x
   
   ### Additional context
   
   `SubqueryExpression.deterministic` reads the same flag, so a scalar or IN subquery over such a
   view is also treated as deterministic under authz (e.g. by `MergeScalarSubqueries`).
   
   A PR with a one-line fix (`override lazy val deterministic = child.deterministic`) and tests follows.
   
   ### Are you willing to submit PR?
   
   - [x] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

Re: [I] [Bug] A nondeterministic permanent view is recomputed for every CTE reference under authz [kyuubi]

["pan3793 (via GitHub)" <gi...@apache.org>]

pan3793 closed issue #7746: [Bug] A nondeterministic permanent view is recomputed for every CTE reference under authz
URL: https://github.com/apache/kyuubi/issues/7746


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org