Re: [PR] [SPARK-56322][CONNECT][PYTHON] Fix TypeError when self-joining observed DataFrames [spark]

["ueshin (via GitHub)" <gi...@apache.org>]

ueshin commented on PR #55140:
URL: https://github.com/apache/spark/pull/55140#issuecomment-4173602105

   cc @herman What's the expected behavior if the same `Observation` is in the both sides of joins?
   
   * `100` because `df` is observed
   * `200` because `df` appears on both sides
   * invalid plan because each `Observation` should appear only once in the whole plan, e.g.
   
   ```py
   >>> df.observe(obs, count(lit(1).alias("row_count")))
   Traceback (most recent call last):
   ...
   pyspark.errors.exceptions.base.PySparkAssertionError: [REUSE_OBSERVATION] An Observation can be used with a DataFrame only once.
   ```
   
   Currently classic returns `100`, but I'm not sure it's correct or now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org